Skip to content

Commit f1f6a11

Browse files
committed
fix(test): make --build-timeout opt-in, and put the e2e's requires on line 2
Two defects in the previous commits, both caught by asking what the change does to projects that are not this one. --build-timeout defaulted to 900s. That would have turned slow-but-CORRECT builds red: one mcpp-index member builds OpenCV from source and measures 1019s on Linux and 1289s on Windows, so a fifteen-minute ceiling fails it and blames mcpp. The asymmetry with --timeout is real and measured, not stylistic — a test binary running over five minutes is unusual, a cold dependency build running over fifteen is ordinary. How long a build may take is a property of the project, so the project says it; mcpp only has to make saying it possible, and that is what was missing. --timeout keeps its 300s default. 178_test_observability.sh carried `# requires: gcc unix-shell` inside the header block. run_all.sh reads that line with `sed -n '2p'`, so it was inert: the test ran on Windows, where <unistd.h> does not exist and where the deadline runner has no kill-by-handle path, and failed for both reasons. Moved to line 2, and narrowed to `unix-shell` — macOS is the platform this file exists for and it has no GCC. The fixture's sleep is now <chrono>/<thread> so the fixture can never be the reason a run fails.
1 parent ac05562 commit f1f6a11

10 files changed

Lines changed: 75 additions & 33 deletions

.agents/docs/2026-07-31-test-observability-implementation-plan.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414

1515
**做法(两条腿,因为 setvbuf 在 Windows 上不成立)**:
1616

17-
1. `main.cpp` 入口 `std::setvbuf(stdout, nullptr, _IOLBF, 0)` —— 覆盖 POSIX,兜住所有不走 ui 的直接输出
17+
1. `mcpp::ui::set_line_buffered()`(POSIX 上 `setvbuf(_IOLBF)`),由 `main()` 调用 —— 兜住所有不走 ui 的直接输出。**不能放 `main.cpp` **:非模块 TU 不允许开 global module fragment 引 `<cstdio>`(Clang 直接拒绝,GCC 静默接受)。**Windows 上不调用**:UCRT`size` 合法范围是 `2..INT_MAX`,0 会走 invalid-parameter handler 直接 abort(0xC0000409 → git-bash 报 exit 127),而 MSVCRT 本来就把 `_IOLBF``_IOFBF`,靠 `ui::flush()` 即可
1818
2. `mcpp::ui` 新增 `flush()`,并在 **每个写 stdout 的 ui 函数**末尾调用(`status` / `info` / `finished` / `plain` / `diagnostic` 的 stdout 分支)。这条在 Windows 上也确定有效,不依赖 `_IOLBF` 语义。
1919
3. `execute.cppm``mcpp test` 的裸 `std::println` 结果行(`... ok` / `FAIL (...)`)改走 `ui::plain`,从而继承 flush。
2020

@@ -37,11 +37,13 @@
3737
**问题**:`--timeout` 只包住测试进程的**运行**。三处 `backend->build()`(`execute.cppm:1056` Phase A / `:1103` bulk / `:1126` per-test)全无期限。macOS 上 `modules/jsc` 卡在 14 次可执行链接上,`--timeout` 设多少都无效。
3838

3939
**做法**:
40-
- `BuildOptions``buildTimeoutSecs`(0 = 不限,默认 **900**)。
40+
- `BuildOptions``buildTimeoutSecs`(0 = 不限,**默认 0 —— 见下**)。
4141
- `ninja_backend``capture_exec` 换成 `capture_exec_deadline`,超时返回 `BuildError{"build timed out after Ns", ...}`
4242
- `run_tests` 三处 build 各自独立计时;超时报成**该成员的构建失败**,扇出继续下一个成员。
4343
- **平台限制如实写明**:`capture_exec_deadline` 在 Windows 上忽略 deadline(`process.cppm:96-99`),因此 `--build-timeout` 目前是 POSIX-only。文档标注,不假装跨平台。
4444

45+
**默认为什么是关的**(与 `--timeout` 不对称,实测而非风格):单个测试跑过 5 分钟不寻常,冷依赖构建跑过 15 分钟很平常 —— mcpp-index 有成员要从源码建 OpenCV,linux 1019s / windows 1289s。默认上限会把「慢但正确」的构建判红。构建能跑多久是工程的性质,由工程来说。
46+
4547
**验收**:e2e —— 一个故意慢的编译边在 `--build-timeout 1` 下被判超时且信息里带成员名。
4648

4749
---
@@ -101,4 +103,4 @@
101103

102104
S1 → S2 → S4 → S5 → S7 → S3 → S6 → S8,每步自带测试。
103105

104-
版本:`2026.7.31.2`(`2026.7.31.1` 已发布)。真源 `src/toolchain/fingerprint.cppm::MCPP_VERSION` + `mcpp.toml`,由 `.github/tools/check_version_pins.sh` 机器校验。
106+
版本:`2026.8.1.1`(`2026.7.31.1` 已发布)。真源 `src/toolchain/fingerprint.cppm::MCPP_VERSION` + `mcpp.toml`,由 `.github/tools/check_version_pins.sh` 机器校验。

CHANGELOG.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,9 @@
2121

2222
- **`mcpp test` 默认有界。** `--timeout` 的默认值从 `0`(不限)改为 **300 秒**;`--timeout 0` 仍然表示不限,只是现在要显式要求。无人值守的 CI 不该被一个挂住的测试吃掉整个 job,而「不限」作为默认值恰恰保证了它可以。
2323

24-
- **`--build-timeout`(默认 900 秒)—— `--timeout` 覆盖不到的另一半。** 此前 `--timeout` 只包住测试**进程的运行**,而 `mcpp test` 的三次 ninja 驱动(包级构建、批量测试构建、逐测试构建)**全都没有期限**。实测一个 macOS lane 卡在某成员的 14 次可执行链接上超过 44 分钟,`--timeout` 设多大都无效。现在每次驱动各自独立计时,超时报成该成员的构建失败并继续下一个成员,失败行明确写「build timeout」而非笼统的 compile 失败。
24+
- **`--build-timeout`(默认关闭)—— `--timeout` 覆盖不到的另一半。** 此前 `--timeout` 只包住测试**进程的运行**,而 `mcpp test` 的三次 ninja 驱动(包级构建、批量测试构建、逐测试构建)**全都没有期限**。实测一个 macOS lane 卡在某成员的 14 次可执行链接上超过 44 分钟,`--timeout` 设多大都无效。现在每次驱动各自独立计时,超时报成该成员的构建失败并继续下一个成员,失败行明确写「build timeout」而非笼统的 compile 失败。
25+
26+
**它与 `--timeout` 不同,默认不开,这个不对称是实测出来的而非风格选择**:单个测试跑过 5 分钟不寻常,而冷依赖构建跑过 15 分钟很平常 —— mcpp-index 有一个成员要从源码建 OpenCV,实测 linux 1019s、windows 1289s。给它一个默认上限会把「慢但正确」的构建判红并让人以为是 mcpp 的错。构建可以跑多久是工程自身的性质,所以由工程来说;mcpp 只需要让「说得出来」成为可能 —— 而这正是原先缺的东西。
2527

2628
平台限制如实说明:deadline 执行器在 Windows 上没有 kill-by-handle 路径,该值被忽略(POSIX only)。
2729

docs/00-getting-started.md

Lines changed: 14 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -87,15 +87,22 @@ mcpp test # compile and run tests/**/*.cpp — one binary per file
8787
mcpp test <pattern> # only tests whose name contains <pattern>
8888
mcpp test --list # enumerate tests without building
8989
mcpp test --timeout 30 # kill a test still RUNNING after 30s (default 300; 0 = no limit)
90-
mcpp test --build-timeout 120 # kill a compile/link still running after 120s (default 900)
90+
mcpp test --build-timeout 120 # kill a compile/link still running after 120s (off by default)
9191
```
9292

93-
`mcpp test` is bounded by default so an unattended CI run cannot be consumed by a
94-
single hung test. The two deadlines cover different halves and neither implies the
95-
other: `--timeout` bounds the test *process*, `--build-timeout` bounds one ninja
96-
drive (the package build, the bulk test build, and each per-test build are timed
97-
separately). A link that never returns is a `--build-timeout` case, not a
98-
`--timeout` one. `--build-timeout` is POSIX-only — the deadline runner has no
93+
The *run* half is bounded by default so an unattended CI job cannot be consumed by
94+
a single hung test. The two deadlines cover different halves and neither implies
95+
the other: `--timeout` bounds the test *process*, `--build-timeout` bounds one
96+
ninja drive (the package build, the bulk test build, and each per-test build are
97+
timed separately). **A link that never returns is a `--build-timeout` case; no
98+
`--timeout` value stops it.**
99+
100+
`--build-timeout` is off by default, and the asymmetry is measured rather than
101+
stylistic: a test binary running over five minutes is unusual, a cold dependency
102+
build running over fifteen is ordinary (one mcpp-index member builds OpenCV from
103+
source in 1019s on Linux and 1289s on Windows). A default ceiling would turn
104+
slow-but-correct builds red. How long a build may take is a property of the
105+
project, so the project says it. POSIX-only — the deadline runner has no
99106
kill-by-handle path on Windows, where the value is ignored.
100107

101108
## Adding Dependencies

docs/06-workspace.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -221,7 +221,7 @@ moment two members both have a `smoke`.
221221

222222
```bash
223223
mcpp test --workspace --timeout 60 # per-test RUN deadline (default 300)
224-
mcpp test --workspace --build-timeout 300 # per-ninja-drive deadline (default 900)
224+
mcpp test --workspace --build-timeout 300 # per-ninja-drive deadline (default 0 = no limit)
225225
mcpp test --workspace --workspace-timeout 1800 # whole fan-out (default 0 = no limit)
226226
```
227227

docs/zh/00-getting-started.md

Lines changed: 11 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -88,14 +88,19 @@ mcpp test # 编译并运行 tests/**/*.cpp —— 每文件一个
8888
mcpp test <pattern> # 只运行名字包含 <pattern> 的测试
8989
mcpp test --list # 只枚举测试,不构建
9090
mcpp test --timeout 30 # 单个测试**运行**超过 30s 被终止(默认 300;0 = 不限)
91-
mcpp test --build-timeout 120 # 单次编译/链接超过 120s 被终止(默认 900)
91+
mcpp test --build-timeout 120 # 单次编译/链接超过 120s 被终止(默认关闭)
9292
```
9393

94-
`mcpp test` 默认是**有界**操作 —— 无人值守的 CI 不该被一个挂住的测试吃掉整个 job。
95-
两个期限覆盖的是不同的一半,互不蕴含:`--timeout` 约束测试**进程的运行**,
96-
`--build-timeout` 约束**单次 ninja 驱动**(包级构建、批量测试构建、每个测试各自
97-
独立计时)。**链接卡死属于 `--build-timeout`,`--timeout` 设多大都无效。**
98-
`--build-timeout` 仅 POSIX 有效 —— Windows 上没有 kill-by-handle 路径,该值被忽略。
94+
**运行**那一半默认有界 —— 无人值守的 CI 不该被一个挂住的测试吃掉整个 job。两个期限
95+
覆盖的是不同的一半,互不蕴含:`--timeout` 约束测试**进程的运行**,`--build-timeout`
96+
约束**单次 ninja 驱动**(包级构建、批量测试构建、每个测试各自独立计时)。
97+
**链接卡死属于 `--build-timeout`,`--timeout` 设多大都无效。**
98+
99+
`--build-timeout` 默认关闭,这个不对称是**实测**出来的而非风格选择:单个测试跑过 5 分钟
100+
不寻常,而冷依赖构建跑过 15 分钟很平常(mcpp-index 有一个成员要从源码建 OpenCV,
101+
linux 1019s、windows 1289s)。给它一个默认上限会把「慢但正确」的构建判红。构建可以跑多久
102+
是工程自身的性质,所以由工程来说。仅 POSIX 有效 —— Windows 上没有 kill-by-handle 路径,
103+
该值被忽略。
99104

100105
## 添加依赖
101106

docs/zh/06-workspace.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -216,7 +216,7 @@ test_paths ... ok (0.31s)
216216

217217
```bash
218218
mcpp test --workspace --timeout 60 # 单测试**运行**期限(默认 300)
219-
mcpp test --workspace --build-timeout 300 # 单次 ninja 驱动期限(默认 900)
219+
mcpp test --workspace --build-timeout 300 # 单次 ninja 驱动期限(默认 0 = 不限)
220220
mcpp test --workspace --workspace-timeout 1800 # 整条扇出(默认 0 = 不限)
221221
```
222222

src/build/execute.cppm

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -806,7 +806,16 @@ export struct TestOptions {
806806
// Per-ninja-invocation deadline (Phase A, the bulk pass, and each per-test
807807
// drive are timed separately). Covers the half `--timeout` never could:
808808
// a compile or link that never returns. POSIX only — see BuildOptions.
809-
int buildTimeoutSecs = 900;
809+
//
810+
// Unlike timeoutSecs this defaults to OFF, and the asymmetry is measured,
811+
// not stylistic. A single test binary running longer than five minutes is
812+
// unusual; a cold dependency build taking longer than fifteen is ordinary —
813+
// one mcpp-index member (OpenCV from source) measures 1019s on Linux and
814+
// 1289s on Windows. A default ceiling would turn those slow-but-correct
815+
// builds red and blame mcpp for it. "How long may a build take" is a
816+
// property of the project, so the project says it; mcpp only has to make
817+
// saying it possible, which is what was missing.
818+
int buildTimeoutSecs = 0;
810819
};
811820

812821
// What one member's `run_tests` actually did. `--workspace` fans out over

src/cli.cppm

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -278,7 +278,7 @@ int run(int argc, char** argv) {
278278
.option(cl::Option("timeout").takes_value().value_name("SECS")
279279
.help("Kill a test still RUNNING after SECS seconds (default 300; 0 = no limit)"))
280280
.option(cl::Option("build-timeout").takes_value().value_name("SECS")
281-
.help("Kill a compile/link drive still running after SECS seconds (default 900; 0 = no limit; POSIX only)"))
281+
.help("Kill a compile/link drive still running after SECS seconds (default 0 = no limit; POSIX only)"))
282282
.option(cl::Option("workspace-timeout").takes_value().value_name("SECS")
283283
.help("Stop the --workspace fan-out after SECS seconds and report what did run (default 0 = no limit)"))
284284
.option(cl::Option("profile").takes_value().value_name("NAME")

tests/e2e/178_test_observability.sh

Lines changed: 12 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
#!/usr/bin/env bash
2+
# requires: unix-shell
23
# 178_test_observability.sh — `mcpp test` is streamed, timed and bounded.
34
#
45
# Four properties, each of which used to be absent and each of which cost a real
@@ -16,9 +17,15 @@
1617
# 4. `--build-timeout` bounds a hung compile/link — the half `--timeout`
1718
# never covered, and the one that actually fires in practice.
1819
#
19-
# See .agents/docs/2026-07-31-test-workspace-observability-analysis.md.
20+
# The `# requires:` line above must stay on line 2 — run_all.sh reads it with
21+
# `sed -n '2p'`, so a requires buried in this block is silently inert (which is
22+
# how an earlier revision of this file ran on Windows and failed there).
23+
# unix-shell, not gcc: macOS is the platform this whole file exists for, and it
24+
# has no GCC. Windows is excluded on purpose — the deadline runner has no
25+
# kill-by-handle path there (mcpp.platform.process), so --timeout and
26+
# --build-timeout are documented POSIX-only and cannot be asserted.
2027
#
21-
# requires: gcc unix-shell
28+
# See .agents/docs/2026-07-31-test-workspace-observability-analysis.md.
2229
set -uo pipefail
2330

2431
TMP=$(mktemp -d)
@@ -40,8 +47,9 @@ cat > fast/tests/quick.cpp <<'EOF'
4047
int main() { return 0; }
4148
EOF
4249
cat > slow/tests/hang.cpp <<'EOF'
43-
#include <unistd.h>
44-
int main() { sleep(30); return 0; }
50+
#include <chrono>
51+
#include <thread>
52+
int main() { std::this_thread::sleep_for(std::chrono::seconds(30)); return 0; }
4553
EOF
4654

4755
# ── 1. streaming: output arrives while the process is still running ───────

tests/unit/test_test_options.cpp

Lines changed: 17 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -6,17 +6,26 @@ import mcpp.build.execute;
66

77
using namespace mcpp::build;
88

9-
// `mcpp test` has to be a BOUNDED operation. It was not: the per-test deadline
10-
// defaulted to 0 (no limit) and no deadline existed for the build drives at all,
11-
// so a single hung test or link could consume an entire unattended CI job with
12-
// nothing to show for it. These defaults are the contract; asserting them here
13-
// means "make it unbounded again" cannot happen by an unnoticed edit.
14-
TEST(TestOptions, DefaultsAreBounded) {
9+
// `mcpp test` has to be BOUNDABLE, and its run half bounded by default. It was
10+
// neither: the per-test deadline defaulted to 0 (no limit) and no deadline
11+
// existed for the build drives at all, so a single hung test or link could
12+
// consume an entire unattended CI job with nothing to show for it. These
13+
// defaults are the contract; asserting them here means neither half can drift
14+
// back by an unnoticed edit.
15+
TEST(TestOptions, RunIsBoundedByDefault) {
1516
TestOptions to;
1617
EXPECT_GT(to.timeoutSecs, 0) << "per-test run deadline must default to a limit";
17-
EXPECT_GT(to.buildTimeoutSecs, 0) << "per-build-drive deadline must default to a limit";
1818
EXPECT_EQ(to.timeoutSecs, 300);
19-
EXPECT_EQ(to.buildTimeoutSecs, 900);
19+
}
20+
21+
// The build deadline deliberately does NOT default on, and the asymmetry is
22+
// measured: a test binary running over five minutes is unusual, a cold
23+
// dependency build running over fifteen is ordinary (one mcpp-index member
24+
// builds OpenCV from source in 1019s on Linux, 1289s on Windows). A default
25+
// ceiling would turn slow-but-correct builds red and blame mcpp for it.
26+
TEST(TestOptions, BuildDeadlineIsOptIn) {
27+
TestOptions to;
28+
EXPECT_EQ(to.buildTimeoutSecs, 0) << "a default build ceiling would fail legitimate cold builds";
2029
}
2130

2231
// 0 is still reachable — "no limit" did not disappear, it just has to be asked

0 commit comments

Comments
 (0)