Skip to content

Commit 3f35d01

Browse files
authored
docs: how to publish a library to mcpp-index, and the generated= path asymmetry (#354)
Two gaps found by walking the chain end to end for libxpkg 0.0.48. Publishing a library was undocumented entirely — `mcpp publish` and `mcpp emit xpkg` appear in the CLI and nowhere in docs/. 09 covers releasing mcpp itself and 02 covers `mcpp pack`; neither says how a library becomes something `[dependencies]` can name. The new 10 writes down the chain and, more usefully, the four places it fails quietly: * gitcode answers HEAD with 401, so an asset verified with HEAD reads as broken and one verified by existence alone can still be the wrong bytes — GET it and cmp it against the GitHub tarball; * an index entry in one platform block resolves on one platform and fails on the others as "no such version", which reads like a consumer typo; * the index is an artifact, so merging to main publishes nothing until publish-artifact.yml runs, and clients hold a TTL on top of that; * a seeded local copy of the unreleased version is indistinguishable from a published one — it is exactly what is still there when the publish failed. The `generated=` row said "relative to the project root". True for the root package; a dependency's build.mcpp resolves the same path against MCPP_OUT_DIR. A library plays both roles, so no relative path is correct in both: writing to MCPP_OUT_DIR and emitting the bare name fails at the root with "declared generated source ... does not exist after the run". Documents the absolute-path form, and that a generated .cppm module interface works — the examples only showed .cpp.
1 parent fd3a0b1 commit 3f35d01

6 files changed

Lines changed: 332 additions & 2 deletions

File tree

docs/07-build-mcpp.md

Lines changed: 31 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,7 @@ is ignored, so you can freely log diagnostics.
4848
| `mcpp:link-lib=<name>` | link `-l<name>` |
4949
| `mcpp:link-search=<dir>` | add a library search dir (`-L`; relative dirs resolve against the project root) |
5050
| `mcpp:cfg=<name>` | define `-D<name>` for both C and C++ |
51-
| `mcpp:generated=<path>` | add a generated source (relative to the project root) to the build |
51+
| `mcpp:generated=<path>` | add a generated source to the build. **A relative path resolves against the project root for the root package, but against `MCPP_OUT_DIR` for a dependency's build.mcpp** — emit an absolute path if the package is both (see below) |
5252
| `mcpp:source=<path>` *(0.0.100+)* | select a **pre-existing** source file into the build (absolute, or relative to the package root). Same downstream effect as `generated=`; use it for files the program *chose* (payload/vendored tree) rather than wrote — e.g. a per-target source selection over a large tarball |
5353
| `mcpp:include-dir=<dir>` *(0.0.100+)* | add a **private** include directory (`-I`) for this package's own TUs (absolute, or relative to the package root; normalized). Replaces the `cxxflag=-I` + `cflag=-I` double emission |
5454
| `mcpp:include-dir-after=<dir>` *(0.0.100+)* | like `include-dir`, but searched **after** the system directories (`-idirafter`) — for payload trees that shadow system headers |
@@ -165,6 +165,36 @@ cache, `MCPP_OUT_DIR`) live in the **consuming project's**
165165
across projects (and may be read-only), so it is never written to; relative
166166
`generated=` paths resolve against `MCPP_OUT_DIR`, not the package root.
167167

168+
### A library that is also built standalone: emit an absolute path
169+
170+
Those two rules — project root for the root package, `MCPP_OUT_DIR` for a
171+
dependency — mean a *relative* `generated=` cannot be right in both roles. A
172+
library is built standalone by its own CI and consumed from the registry by
173+
everyone else, so it plays both.
174+
175+
Writing into `MCPP_OUT_DIR` and emitting the bare filename works as a
176+
dependency and fails at the root with:
177+
178+
```
179+
error: build.mcpp declared generated source 'foo.cppm' but it does not exist after the run
180+
```
181+
182+
Write to `MCPP_OUT_DIR` (the package root may be read-only) and emit the
183+
**absolute** path:
184+
185+
```cpp
186+
const auto out = std::filesystem::path(mcpp::out_dir()) / "foo.cppm";
187+
// ... write it ...
188+
mcpp::generated(out.string().c_str());
189+
```
190+
191+
`mcpp::out_dir()` is always absolute, so this is correct in both roles and
192+
needs no branch on which one you are in.
193+
194+
A generated **module interface** is fine here: `.cppm` goes through the same
195+
scan as any other source, so a generated file declaring `export module …` can
196+
be imported by the package's own TUs.
197+
168198
## Incremental: declared inputs (no needless re-runs)
169199
170200
mcpp does **not** re-run `build.mcpp` on every build. It caches the program's

docs/10-publishing-a-library.md

Lines changed: 142 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,142 @@
1+
# 10 - Publishing a Library to mcpp-index
2+
3+
**English** | [简体中文](zh/10-publishing-a-library.md)
4+
5+
How a library becomes something `[dependencies]` can name. This is the
6+
*library author's* chain; [09 - Releasing mcpp](09-release.md) is about
7+
releasing mcpp itself, and [02 - Packaging & Release](02-pack-and-release.md)
8+
is about `mcpp pack` bundling an application.
9+
10+
## The chain, in the only order that works
11+
12+
```
13+
your repo merge → git tag → GitHub auto-generates the tag tarball
14+
15+
gitcode mirror a byte-identical copy, for the CN region
16+
17+
mcpplibs/mcpp-index pkgs/<x>/<name>.lua — GLOBAL + CN URLs + sha256
18+
↓ publish-artifact.yml pushes a content-hash artifact
19+
20+
consumers bump the version in their mcpp.toml
21+
```
22+
23+
Each arrow is a gate. Skipping one does not fail loudly — it fails as
24+
"dependency not found" or a 404 in someone else's build, hours later.
25+
26+
## 1. Tag the release
27+
28+
The version in `mcpp.toml` and the tag must agree. GitHub generates
29+
`archive/refs/tags/<tag>.tar.gz` automatically; that tarball **is** the
30+
artifact — nothing needs uploading.
31+
32+
```bash
33+
git tag 0.0.48 && git push origin 0.0.48
34+
curl -fsSL -o pkg-0.0.48.tar.gz \
35+
https://github.com/<owner>/<repo>/archive/refs/tags/0.0.48.tar.gz
36+
sha256sum pkg-0.0.48.tar.gz # ← the digest the index will carry
37+
```
38+
39+
The tarball extracts to `<repo>-<tag>/`, and mcpp looks for `mcpp.toml` inside
40+
that wrapper directory. A repo that ships its own `mcpp.toml` needs no `mcpp`
41+
field in the index entry.
42+
43+
## 2. Mirror to gitcode
44+
45+
The CN entry must be a **byte-identical copy** of the GitHub tarball, only
46+
renamed. Anything else and the two regions disagree about what a pinned
47+
`sha256` means.
48+
49+
```bash
50+
gtc release publish mcpp-res/<name> --tag 0.0.48 --asset pkg-0.0.48.tar.gz
51+
```
52+
53+
Then verify it, because the upload reporting success is not the same as the
54+
asset being fetchable:
55+
56+
```bash
57+
# GET, never HEAD — gitcode answers HEAD with 401 and GET with 302 → CDN 200
58+
curl -fsSL -o cn.tar.gz \
59+
https://gitcode.com/mcpp-res/<name>/releases/download/0.0.48/pkg-0.0.48.tar.gz
60+
cmp cn.tar.gz pkg-0.0.48.tar.gz # must be identical, not merely present
61+
```
62+
63+
## 3. Add the index entry
64+
65+
In `mcpplibs/mcpp-index`, `pkgs/<first-letter>/<name>.lua`:
66+
67+
```lua
68+
["0.0.48"] = {
69+
url = {
70+
GLOBAL = "https://github.com/<owner>/<repo>/archive/refs/tags/0.0.48.tar.gz",
71+
CN = "https://gitcode.com/mcpp-res/<name>/releases/download/0.0.48/pkg-0.0.48.tar.gz",
72+
},
73+
sha256 = "<the digest from step 1>",
74+
},
75+
```
76+
77+
**In all three platform blocks**`linux`, `macosx`, `windows`. A source
78+
tarball is the same bytes on every platform, and an entry present in only one
79+
of them fails on the others as "no such version", which reads like a typo in
80+
the consumer's manifest.
81+
82+
## 4. Wait for the artifact
83+
84+
**The index is an artifact, not a git clone.** Merging to `main` is not
85+
enough: `publish-artifact.yml` has to run and push a content-hash artifact,
86+
and clients hold a refresh TTL on top of that. Editing a cached `pkgs/**` by
87+
hand does nothing.
88+
89+
```bash
90+
gh run list --repo mcpplibs/mcpp-index --workflow publish-artifact.yml --limit 1
91+
rm -rf ~/.mcpp/registry/data/<namespace> # force a client refresh
92+
```
93+
94+
## 5. Verify from a cold resolve, then bump consumers
95+
96+
The point of this step is that a local checkout of the library will mask every
97+
mistake above. Resolve it the way a stranger would:
98+
99+
```bash
100+
rm -rf ~/.mcpp/registry/data/xpkgs/<ns>-x-<name>/0.0.48
101+
mcpp build # must download and compile 0.0.48
102+
```
103+
104+
Only then bump `[dependencies]` in the consumers.
105+
106+
> Do **not** try to force a refresh with
107+
> `find ~/.mcpp/registry -mindepth 1 -maxdepth 1 ! -name data -exec rm -rf {} +`
108+
> alone. `data/xpkgs` sits at depth 2 and is not named `data`, so a careless
109+
> second pass deletes the whole payload store (~800 MB of toolchains).
110+
> Recovery is `mcpp self doctor`, which re-provisions, then `mcpp update`.
111+
112+
## Testing against an unreleased version
113+
114+
While the chain above is still in flight, seed the registry by hand so
115+
consumers can compile against the library before it is published:
116+
117+
```bash
118+
REG=~/.mcpp/registry/data/xpkgs/<ns>-x-<name>/0.0.48
119+
mkdir -p "$REG"
120+
git -C /path/to/library archive --format=tar --prefix=<repo>-0.0.48/ HEAD \
121+
| tar -x -C "$REG"
122+
touch "$REG/.mcpp_ok" # the marker that says "resolved"
123+
cp ../0.0.47/.xpkg.lua "$REG/.xpkg.lua" # add a 0.0.48 entry to it
124+
```
125+
126+
mcpp's build sandbox is network-isolated, so `file://` and
127+
`http://127.0.0.1` index URLs cannot be fetched — seeding the cache is the way.
128+
129+
**Remove the seeded copy before believing the real thing works.** A seeded
130+
0.0.48 and a published 0.0.48 are indistinguishable to the build, and the
131+
seeded one is the copy that will still be there when the publish silently
132+
failed.
133+
134+
## Checklist
135+
136+
- [ ] `mcpp.toml` version == git tag
137+
- [ ] tag pushed; tarball downloads and its sha256 recorded
138+
- [ ] gitcode asset verified with **GET**, byte-identical to GitHub's
139+
- [ ] index entry in **all three** platform blocks
140+
- [ ] `publish-artifact.yml` succeeded
141+
- [ ] cold resolve (seeded copy deleted) downloads and compiles it
142+
- [ ] consumers bumped

docs/README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@
1212
- [07 - build.mcpp Build Program](07-build-mcpp.md)
1313
- [08 - Toolchain Internals](08-toolchain-internals.md)
1414
- [09 - Releasing mcpp](09-release.md)
15+
- [10 - Publishing a Library to mcpp-index](10-publishing-a-library.md)
1516

1617
## Specifications
1718

docs/zh/07-build-mcpp.md

Lines changed: 26 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,7 @@ mcpp build # 编译 + 运行 build.mcpp,然后构建工程
4545
| `mcpp:link-lib=<name>` | 链接 `-l<name>` |
4646
| `mcpp:link-search=<dir>` | 增加库搜索目录(`-L`;相对路径按工程根目录解析) |
4747
| `mcpp:cfg=<name>` | 为 C 与 C++ 同时定义 `-D<name>` |
48-
| `mcpp:generated=<path>` | 把生成的源码(相对工程根目录)加入构建 |
48+
| `mcpp:generated=<path>` | 把生成的源码加入构建。**相对路径在根工程按工程根解析,在依赖的 build.mcpp 里按 `MCPP_OUT_DIR` 解析** —— 两种角色都可能出现的包应发绝对路径(见下文) |
4949
| `mcpp:source=<path>` *(0.0.100+)* | 把一份**既有**源文件选入构建(绝对路径,或相对包根)。下游效果与 `generated=` 相同;语义区别在于文件是程序*选中*的(tarball payload / vendored 源树)而非程序写出的——例如对大型源码包做 per-target 源选择 |
5050
| `mcpp:include-dir=<dir>` *(0.0.100+)* | 为本包自身 TU 增加一个**私有** include 目录(`-I`;绝对路径或相对包根,自动规范化)。取代过去 `cxxflag=-I` + `cflag=-I` 的双重裸发 |
5151
| `mcpp:include-dir-after=<dir>` *(0.0.100+)* |`include-dir`,但排在系统目录**之后**搜索(`-idirafter`)——用于会遮蔽系统头的 payload 源树 |
@@ -152,6 +152,31 @@ mcpp 会把它自己构建时用的**同一份** std 模块暂存过来,缓存
152152
`target/.build-mcpp/deps/<pkg>@<ver>/` 下——registry 包根跨工程共享(且可能只读),
153153
绝不写入;相对 `generated=` 路径按 `MCPP_OUT_DIR` 解析,而非包根。
154154

155+
### 既独立构建又被当依赖的库:发绝对路径
156+
157+
上面这两条规则——根工程按工程根、依赖按 `MCPP_OUT_DIR`——意味着**相对**
158+
`generated=` 不可能两种角色都对。而一个库正好两种角色都有:自己的 CI 独立构建它,
159+
别人从 registry 当依赖用它。
160+
161+
写进 `MCPP_OUT_DIR` 再发裸文件名,在依赖角色下能用,在根工程下则失败:
162+
163+
```
164+
error: build.mcpp declared generated source 'foo.cppm' but it does not exist after the run
165+
```
166+
167+
正确做法是写进 `MCPP_OUT_DIR`(包根可能只读),并发**绝对**路径:
168+
169+
```cpp
170+
const auto out = std::filesystem::path(mcpp::out_dir()) / "foo.cppm";
171+
// ... 写文件 ...
172+
mcpp::generated(out.string().c_str());
173+
```
174+
175+
`mcpp::out_dir()` 恒为绝对路径,因此两种角色下都正确,不需要判断自己处在哪一种。
176+
177+
生成**模块接口**是可以的:`.cppm` 走与其他源文件相同的扫描,所以一个生成出来的、
178+
声明 `export module …` 的文件可以被该包自己的 TU import。
179+
155180
## 增量:声明输入(避免无谓重跑)
156181
157182
mcpp **不会**每次构建都重跑 `build.mcpp`。它会缓存程序产出的指令,只有当它依赖的东西

docs/zh/10-publishing-a-library.md

Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,131 @@
1+
# 10 - 发布一个库到 mcpp-index
2+
3+
[English](../10-publishing-a-library.md) | **简体中文**
4+
5+
一个库如何变成 `[dependencies]` 可以写出来的东西。这是**库作者**的链路;
6+
[09 - 发布 mcpp](09-release.md) 讲的是发布 mcpp 自身,
7+
[02 - 发布打包](02-pack-and-release.md) 讲的是 `mcpp pack` 打包应用。
8+
9+
## 这条链,只有一种顺序成立
10+
11+
```
12+
你的仓库 merge → git tag → GitHub 自动生成 tag tarball
13+
14+
gitcode 镜像 逐字节相同的一份拷贝,供 CN 区
15+
16+
mcpplibs/mcpp-index pkgs/<x>/<name>.lua —— GLOBAL + CN 双 URL + sha256
17+
↓ publish-artifact.yml 推出内容哈希 artifact
18+
19+
消费方 在自己的 mcpp.toml 里升版本
20+
```
21+
22+
每一支箭头都是一道关。漏掉任何一道都不会当场报错 —— 它会在几小时后,
23+
以别人构建里的 "dependency not found" 或一个 404 的形式出现。
24+
25+
## 1. 打 tag
26+
27+
`mcpp.toml` 里的版本必须与 tag 一致。GitHub 会自动生成
28+
`archive/refs/tags/<tag>.tar.gz`,**那个 tarball 就是产物**,不需要另行上传。
29+
30+
```bash
31+
git tag 0.0.48 && git push origin 0.0.48
32+
curl -fsSL -o pkg-0.0.48.tar.gz \
33+
https://github.com/<owner>/<repo>/archive/refs/tags/0.0.48.tar.gz
34+
sha256sum pkg-0.0.48.tar.gz # ← 索引里要写的摘要
35+
```
36+
37+
tarball 解开是 `<repo>-<tag>/`,mcpp 会在这层包装目录里找 `mcpp.toml`
38+
仓库自带 `mcpp.toml` 时,索引条目不需要 `mcpp` 字段。
39+
40+
## 2. 镜像到 gitcode
41+
42+
CN 条目必须是 GitHub tarball 的**逐字节拷贝**,只改文件名。否则两个区域对同一个
43+
`sha256` 的理解就不一致了。
44+
45+
```bash
46+
gtc release publish mcpp-res/<name> --tag 0.0.48 --asset pkg-0.0.48.tar.gz
47+
```
48+
49+
然后**验证**它 —— 上传报成功和资源真的能取到,不是一回事:
50+
51+
```bash
52+
# 用 GET,绝不用 HEAD —— gitcode 对 HEAD 返回 401,对 GET 返回 302 → CDN 200
53+
curl -fsSL -o cn.tar.gz \
54+
https://gitcode.com/mcpp-res/<name>/releases/download/0.0.48/pkg-0.0.48.tar.gz
55+
cmp cn.tar.gz pkg-0.0.48.tar.gz # 必须一致,而不只是"存在"
56+
```
57+
58+
## 3. 加索引条目
59+
60+
`mcpplibs/mcpp-index``pkgs/<首字母>/<name>.lua`:
61+
62+
```lua
63+
["0.0.48"] = {
64+
url = {
65+
GLOBAL = "https://github.com/<owner>/<repo>/archive/refs/tags/0.0.48.tar.gz",
66+
CN = "https://gitcode.com/mcpp-res/<name>/releases/download/0.0.48/pkg-0.0.48.tar.gz",
67+
},
68+
sha256 = "<第 1 步得到的摘要>",
69+
},
70+
```
71+
72+
**三个平台块都要写** —— `linux``macosx``windows`。源码 tarball 在每个平台上是
73+
同样的字节;只写了其中一个,在另外两个平台上会以 "no such version" 失败,
74+
而那读起来像是消费方 manifest 里打错了字。
75+
76+
## 4. 等 artifact
77+
78+
**索引是 artifact,不是 git clone。** 合进 `main` 还不够:必须等
79+
`publish-artifact.yml` 跑完并推出内容哈希 artifact,客户端之上还有一层刷新 TTL。
80+
手改缓存里的 `pkgs/**` 不起任何作用。
81+
82+
```bash
83+
gh run list --repo mcpplibs/mcpp-index --workflow publish-artifact.yml --limit 1
84+
rm -rf ~/.mcpp/registry/data/<namespace> # 强制客户端刷新
85+
```
86+
87+
## 5. 冷解析验证,然后再升消费方
88+
89+
这一步的意义在于:本地那份库的 checkout 会掩盖上面每一个错误。要像一个陌生人那样解析它:
90+
91+
```bash
92+
rm -rf ~/.mcpp/registry/data/xpkgs/<ns>-x-<name>/0.0.48
93+
mcpp build # 必须真的下载并编译 0.0.48
94+
```
95+
96+
通过之后,才去升消费方的 `[dependencies]`
97+
98+
> **不要**只用
99+
> `find ~/.mcpp/registry -mindepth 1 -maxdepth 1 ! -name data -exec rm -rf {} +`
100+
> 去强制刷新。`data/xpkgs` 在第 2 层且名字不是 `data`,再来一遍粗心的清理就会把整个
101+
> payload 仓(约 800 MB 工具链)删掉。恢复办法是 `mcpp self doctor` 重新 provision,
102+
> `mcpp update`
103+
104+
## 对着尚未发布的版本做测试
105+
106+
在上面这条链还没走完时,可以手工播种 registry,让消费方提前编译:
107+
108+
```bash
109+
REG=~/.mcpp/registry/data/xpkgs/<ns>-x-<name>/0.0.48
110+
mkdir -p "$REG"
111+
git -C /path/to/library archive --format=tar --prefix=<repo>-0.0.48/ HEAD \
112+
| tar -x -C "$REG"
113+
touch "$REG/.mcpp_ok" # 表示"已解析"的标记
114+
cp ../0.0.47/.xpkg.lua "$REG/.xpkg.lua" # 往里加一条 0.0.48 条目
115+
```
116+
117+
mcpp 的构建沙箱是网络隔离的,`file://``http://127.0.0.1` 形式的索引 URL 取不到,
118+
播种缓存才是可行的办法。
119+
120+
**在相信"真的能用"之前,先把播种的那份删掉。** 播种的 0.0.48 和已发布的 0.0.48
121+
对构建来说毫无区别 —— 而当发布其实失败了的时候,留在那里的正是播种的那一份。
122+
123+
## 检查清单
124+
125+
- [ ] `mcpp.toml` 版本 == git tag
126+
- [ ] tag 已推;tarball 可下载,sha256 已记录
127+
- [ ] gitcode 资源用 **GET** 验证过,且与 GitHub 那份逐字节一致
128+
- [ ] 索引条目写进了**三个**平台块
129+
- [ ] `publish-artifact.yml` 成功
130+
- [ ] 冷解析(删掉播种拷贝后)能下载并编译
131+
- [ ] 消费方已升版本

docs/zh/README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,3 +12,4 @@
1212
- [07 - build.mcpp 构建程序](07-build-mcpp.md)
1313
- [08 - 工具链机制内幕](08-toolchain-internals.md)
1414
- [09 - 发布 mcpp](09-release.md)
15+
- [10 - 发布一个库到 mcpp-index](10-publishing-a-library.md)

0 commit comments

Comments
 (0)