Skip to content

Every open bug, plus four high-priority fixes (compile time, ubuntu 26.04, script gateway, reference docs) - #3502

Merged
nicolas-maman merged 17 commits into
mainfrom
fix/nov01-batch10
Nov 1, 2026
Merged

nicolas-maman merged 17 commits into
mainfrom
fix/nov01-batch10

Conversation

@nicolas-maman

@nicolas-maman nicolas-maman commented Nov 1, 2026 •

Copy link
Copy Markdown
Collaborator

Every open bug, and four high-priority issues the owner chose for this batch.

Scope

Fixed at the start. Anything found on the way that is outside it is filed for the next batch, unless it is a security, crash or data-loss problem.

Bugs:

Four more, critical or high priority:

What changed

#3499, compile time. The release's macOS x86_64 leg failed twice on order_effects_scaling: 8000 chained functions took 22 to 25 s to compile, against a 20 s bound. Sampling the compile with gdb found four lookups by name that scanned the whole program, once per call or once per top-level node:

  • the checker's callee and extern lookups;
  • codegen's function-pointer globals;
  • codegen's set of function names a local may shadow;
  • the merge's import prefixes.

Each is now answered from an index the compiler already keeps: the checker's clause index, codegen's program index, or a hashed name set.

functions before after
4000 282 ms 170 ms
8000 957 ms 325 ms
16000 3691 ms 651 ms
32000 about 20 s 1319 ms

The test now compares 4000 and 16000 functions on the same runner instead of a wall-clock bound: four times the functions must cost less than eight times the time. Main's compiler fails it at 13 times; this one passes at 3.7.

#3478. The expression form of if parsed its condition without the guard the if statement, while and match use, so a call at its end took the then arm's { as a trailing block. A call inside an arm still takes one. A call's arguments now parse with the guard cleared, since inside the parentheses a { cannot open the body: a struct literal argument in a condition was refused, in the if statement, while and match too.

#3479. s[i] on a string had no type, so a local bound to it was left to codegen's int default. Two more defects in the same expression:

  • The C read a plain char, which is signed on x86: a byte past 127, any non-ASCII UTF-8 byte, read negative there and positive on ARM.
  • Nothing checked the index: s[100000] on "Az" read past the string, a memory-safety defect, fixed here for that reason.

s[i] is now an int from 0 to 255, the type codegen always gave such a local and the value string.bytes(s)[i] has. s[length] is the terminator, 0, as a C-style scan reads it. An index below 0 or past the terminator is the panic string index i out of range for length n, which a try catches, as a slice index's is. (A byte type, the first version here, refused uint32 u = s[i] and wrapped c - 100; the review caught it.)

#3485. The refusals called exit(1) from inside the stub writer, with the stub file still open. C runs atexit handlers before it closes streams, and Windows will not delete an open file, so every refused build left an ae-binimport-* directory: 1,168 on one machine. The refusals now return to the caller, which closes and removes the stub before exiting. #3486 removes the unused function.

#3483 and #3488, both timing-bound tests:

  • Ask relief now holds B's reply until D has handled its 20 messages, instead of pausing 400 ms. A relief that does not run D still fails it. The test frees its actors before asserting, so a failure leaks nothing.
  • The account-free scaling test times the free by the thread's CPU clock on POSIX, the best of five runs, so a preempted run no longer reads as superlinear.

#3500.

  • Every ubuntu-latest job now names ubuntu-26.04. GitHub moves the label between Oct 19 and Nov 19, so until then a job could land on either image from one run to the next. Paul's 4861daa already fixed the qemu package.
  • The actions GitHub flags as Node 20 move to the releases that declare node24: checkout v5, cache v5, upload-artifact v6, download-artifact v7, setup-emsdk v15, action-gh-release v3, setup-java v5, setup-go v6.
  • mlugg/setup-zig and ilammy/msvc-dev-cmd have no Node 24 release yet; GitHub runs them on Node 24.
  • The release workflow's own jobs only run after a merge. Its two cross builds are the same steps ci.yml's FreeBSD and Android jobs run here on 26.04.

#3501. The owner chose Windows' rule for POSIX. script_gateway.mount reads which image gives the script its runtime (dlsym on the script's handle, dladdr), against the image that holds the host's:

  • Mounted: a script with no runtime of its own, or one on the host's runtime.
  • Refused, naming --shared-runtime: a script that carries its own.
  • A --shared-runtime script on a static host: mounted on ELF, where the global scope binds it to the program's runtime; refused on macOS, where each import binds to its library.

The docs build scripts with --shared-runtime.

#3480.

  • A handler that returns without a reply ends the ask at once.
  • Constant-time comparison points to ct.bytes_eq and ct.bytes_eq_slice.
  • The call deadline is one instant per thread, not a per-call budget.
  • The memory cap is process-wide.

Found by this PR's CI and fixed here

The benchmarks workflow ran because this PR changes it, and every Aether entry was "did not build" on Linux. The link compiled the runtime from source with -flto beside the whole libaether.a. GCC's LTO plugin does not list what top-level asm defines, and the rendezvous records from #3445 are such asm. So the link took the archive's aether_resource_caps.o for its anchor, and every symbol in it was then defined twice. It has been broken on main since #3445; the workflow runs only on benchmark changes and on Mondays. std now comes from a copy of libaether.a with the runtime members the MANIFEST lists taken out. The benchmark run on b9709af passed.

Reviews

One fresh review of the whole diff found six things, all fixed:

  • the gateway's POSIX check broke the wasm32-wasi build: there is no dladdr there, and no dynamic loader either, so there is no check;
  • the rewritten ask-relief test could deadlock: with B on core 1 the relief moved D behind B's own step, which waited for D. B is now on core 2, as in the file's overflow cases;
  • s[i] as a byte refused or changed programs that compiled, and the bounds check broke s[length] scans: it is an int now, as above;
  • a struct literal argument in an if expression's condition was refused (An if used as a value does not parse when its condition ends in a call: s = if f(x) { a } else { b } #3478, as above);
  • the docs overstated what a static macOS host refuses;
  • a strlen per s[i] on a C string without a length header. This is left as is: string.length(s) on such a string already costs the same in a loop's condition.

While fixing them, a main bug turned up and is fixed here, since it ends the process: a return inside a try with no defers popped the try frames before computing its value. So try { return xs[i] } catch e { ... }, or a returned call that panics, escaped the try. The value is now computed inside the frames, for a single value, a tuple and main's return, as the defer-aware path already did (test_return_inside_try_catches).

The review of those fixes, the last one, found four of the five correct, checked with probes for every return type, nested trys, and conditions with nested calls and struct literals. It found one regression, now fixed:

  • main ran its defers before computing its return value. That was an older bug: return string.length(s) read a released s. With the panic now caught, main also went on with its defers already run, and ran them twice. main now computes the value first, as every other function does (test_main_return_inside_try, which fails before the fix).
  • On WASI, where a script's runtime cannot be read, the gateway now refuses rather than mounts. This is unreachable today, since the load fails first.

Also from CI: order_effects_scaling on macOS ARM64 measured one 16000-function compile at 14 times one of 4000, where the same commit measures 3.7x here and the leg passed one commit before. The test now takes the best of three runs of each size, interleaved.

Checks

Local, Windows x86_64 (MSYS2 MINGW64):

  • every emit_lib_* and binary_import_* integration test, abi_policy, sandbox_nested_libraries, order_effects_scaling (fails on main), http_script_gateway;
  • the C runner 653/653, three runs alike;
  • the full make test-ae sweep on cf912d7: 1980 passed, 0 failed, 108 skipped (Windows skips); again on the final head before merge;
  • check-docs, check-api-docs, check-changelog, check-tests, check-examples;
  • the regressions test_if_expression_call_condition, test_string_index_byte, test_return_inside_try_catches and test_main_return_inside_try.

The changed C compiles with -Werror for x86_64 Linux, aarch64 macOS and FreeBSD (zig).

Closes #3478, closes #3479, closes #3480, closes #3483, closes #3485
Closes #3486, closes #3488, closes #3499, closes #3500, closes #3501

The release's macOS x86_64 leg failed twice on order_effects_scaling
(8000 chained functions in 22-25 s against a 20 s bound). Sampled with
gdb, the compile was quadratic through four lookups by name that scanned
the program per call or per top-level node:

- typechecker effect_find_func and call_extern_decl (every call's
  declaration, from the check_frees and @keep walks): now answered from
  the checker's clause index, which also indexes externs;
- codegen lookup_fnptr_global (every call): now in the program index;
- codegen rename_locals_shadowing_functions: its NameSet was a list
  scanned per reference; now a StrMap, and externs by name;
- module merge rewrite_import_prefixes: the program's imports were
  collected again for each top-level node; now once.

8000 functions: 957 -> 325 ms here; 16000: 3691 -> 651 ms; 32000:
about 20 s -> 1.3 s. The test now compares 4000 and 16000 functions on
the same runner (linear is 4x; main's compile was 13x, which fails it).
…#3500)

GitHub moves ubuntu-latest from 24.04 to 26.04 between 2026-10-19 and
2026-11-19, so until then a job lands on either image from one run to the
next (contrib.sqlite's cross build failed on a 26.04 runner and passed on
the next run; Paul's 4861daa fixed its package). The jobs that used
ubuntu-latest (ci.yml six, memory-check.yml, windows.yml two, release.yml
six, changelog-collect.yml) now name ubuntu-26.04, so each run gets the
image this PR checks them on. The release's Linux build stays on 22.04
for its glibc floor.

The actions GitHub flags as Node 20 move to the releases that declare
node24: checkout v5, cache restore/save v5, upload-artifact v6,
download-artifact v7, setup-emsdk v15, action-gh-release v3, and in the
benchmarks setup-java v5 and setup-go v6. mlugg/setup-zig and
ilammy/msvc-dev-cmd have no node24 release yet and stay (GitHub runs them
on Node 24).
…3478)

The expression form of `if` parsed its condition without the in_condition
guard the if statement, while and match use, so a call at its end took
the then arm's `{` as a trailing block ("Expected LEFT_BRACE, got
ELSE"). The condition is parsed under the guard and the arms without it,
so a call inside an arm still takes a trailing block. Regression test
covers a lone call, a call ending an &&, an else-if chain of calls, and a
trailing closure in an arm.
The checker gave a string index no type, so a local bound to it warned
"unresolved type in codegen, defaulting to int". The generated C read a
plain char (signed on x86: a UTF-8 byte past 127 read negative there and
positive on ARM), and nothing checked the index: s[100000] on "Az" read
past the string. The read is now typed byte in the inference pass and the
checker, the value string.bytes(s)[i] has, and codegen reads it through
_aether_str_at, which checks it against the string's length and panics
"string index i out of range for length n" (aether_string_fail_index),
as a slice index does. The language reference says so.
The refusals (ae_lib_check_compat, ae_refuse_split_ownership) called
exit(1) from inside ae_generate_binimport_stub, with the stub's FILE still
open. C runs atexit handlers before it closes streams, so the cleanup that
removes the stub directory ran against an open file, which Windows will
not delete: every refused build left ae-binimport-<pid>-<n> behind (1,168
on one machine). The refusals now return 2; the caller closes the stub,
removes it and exits, and the atexit cleanup finds nothing open. The
ownership test runs each refused build in a temp directory of its own and
checks it is empty after (failed before, on Windows).

Also removes ae_source_module_exists, which nothing calls (#3486): clang's
-Wall -Werror refused the file for it.
The benchmarks compile the runtime from source with -O3 -march=native
-flto and took std from the whole libaether.a. On ELF the link then
defined the runtime twice: GCC's LTO plugin does not list what top-level
asm defines, and the rendezvous records are such asm (#3445,
runtime/aether_rv_mark.h), so the link took the archive's
aether_resource_caps.o for g_accounts_mark beside the compiled one. Every
Aether entry has been 'did not build' on Linux since #3445 (the workflow
runs on benchmark changes and on Mondays; this PR's workflow bump ran it).
std now comes from a copy of libaether.a with the runtime members the
MANIFEST lists taken out, so the runtime is linked once, still with the
benchmark's own flags.
'The waiting core's other actors run' had B pause 400 ms and counted
what D, on A's waiting core, handled meanwhile: on a slow runner (macOS
ASan, Valgrind) the pause ended with 11 of the 20 handled. B now holds
its reply until D has handled its 20 (bounded by 3 s, as the overflow
cases in the file hold theirs), and D's messages go out once B has A's
ask, so A is waiting. A relief that does not run D still fails it: B
answers after 3 s with D's count short. The checks read the counts
first and free the actors before asserting, so a failure leaks nothing
(Valgrind reported them definitely lost). 47-62 ms here instead of 416.
accounts_free_is_linear_in_its_blocks compared wall-clock frees of
10,000 and 40,000 blocks, and on a loaded macOS ASan runner one 40,000
free took 22 times one of 10,000 (5.13 against 0.23 ms): the clock
counted time the thread was not running. The free's cost is CPU work, so
on POSIX it is now timed by CLOCK_THREAD_CPUTIME_ID, the best of five
runs instead of three. Windows keeps the performance counter (its thread
times tick at 15.6 ms). Here: 0.12 against 0.48 ms, three runs alike.
- language reference, Reply Statement: a handler that returns without a
  reply ends the ask at once (AETHER_ASK_NO_REPLY, "the handler returned
  without replying"); the timeout is for one that has not returned.
- stdlib reference, std.cryptography: constant-time comparison points to
  ct.bytes_eq and ct.bytes_eq_slice, which exist.
- emit-lib.md and libaether.h: the call deadline is one instant per
  thread, now + ms when set, cutting every later call until a fresh one
  or 0, not a per-call budget; and the memory cap is process-wide, not
  one of two TLS knobs.
…3501)

Since #3481 a script built the default way keeps its own runtime on every
platform; Windows refused that at mount, POSIX mounted it unchecked: two
runtimes, each with its own caps, config and panic frames, while the
script filled objects the host's runtime frees. The owner chose the
Windows rule for POSIX.

sg_script_runtime_check now runs everywhere. On POSIX it reads which
image gives the script's handle aether_runtime_abi (dlsym, dladdr)
against the image holding the host's own:
- none (runtime calls left unresolved, bound to the host at dlopen) or
  the host's: one runtime, mounted;
- the script's own image: refused, KIND_IO, naming --shared-runtime;
- the shared runtime while the host is static: mounted on ELF, where the
  global scope binds the script to the -rdynamic program's runtime, and
  refused on macOS, where each import binds to its library.

The gateway test gains the POSIX cases (an own-runtime script refused; a
--shared-runtime script on the static host mounts on ELF and is refused
on macOS). The module docs, the header and the stdlib reference build
scripts with --shared-runtime.
… refuses (#3501)

The review of the batch: wasm32-wasi has <dlfcn.h> but no Dl_info or
dladdr, so the POSIX branch broke the WebAssembly job. WASI has no
dynamic loader (a mount fails at the load), so the check is none there.
The docs said a static macOS host is refused; it is refused only for a
--shared-runtime script, and a script with no runtime of its own mounts.
…3483)

The review of the batch: with B on core 1 the relief could move D to
B's core (relief_destination ranks cores by queued work, and a core in a
long step with an empty queue looks idle), behind B's step, which waited
for D: three CI legs saw A's ask time out. B is now on core 2, as in the
overflow cases, so core 1 is the relief's choice for D, and B holds at
most 2 s, inside A's 3 s timeout, so a relief that fails shows as a short
count rather than a timeout.
…nator (#3479)

The review of the batch: typed byte, s[i] refused uint32 u = s[i] and
the uint8/uint16 widths, and wrapped c = s[i]; c = c - 100 (205 for -51);
and the bounds check panicked on s[length], which a C-style scan
(while s[j] != 0) and s[0] on an empty or null string read. The read is
now an int, the type codegen always gave such a local, still unsigned
(0 to 255, string.bytes(s)[i]'s value); index length reads 0 (returned,
not loaded); below 0 or past it panics. The regression test covers the
terminator, a scan, the empty string, one past the terminator, a far
index and a negative one.
The review of the batch: under the condition guard a struct literal
among a call's arguments was refused ("Expected RIGHT_PAREN, got
LEFT_BRACE"), new for the if expression and long standing for if,
while and match. Inside the parentheses a `{` cannot open the body, so
the arguments parse with the guard cleared, restored at the `)` before
the trailing-block check (and on every error path).
Found while fixing #3479's review: a return with no defers to run
popped the try frames before it computed the value, so
try { return xs[i] } catch e { ... } with i out of range, or a returned
call that panics, ended the process ("panic outside any try/catch").
The single-value and tuple paths now compute into a temporary inside the
frames and pop after, as the defer-aware path already did, and main's
return assigns main_exit_ret before its pops. Regression test: an index,
a panicking call and a tuple, in range and out.
The review of the fix round: main ran its defers (the program's and the
release of its owned locals) before computing its return value. A value
reading a released local read freed memory (return string.length(s)
exited 0, not 3), and now that a panic in the value is caught by a try
around it, main went on with its defers run and its locals released,
and ran the defers again at its end. main_exit_ret is computed first,
then the defers, then the try pops, as every other function's return.
Regression test: a defer that must run once, a local read after the
caught panic, and an exit code read from a local.

Also from that review: on WASI, where no runtime can be read, the
gateway refuses a script rather than mount it unchecked (fail closed;
unreachable today, as the load fails first).
…#3499)

macOS ARM64 measured a single 4000-function compile at 220 ms and a
single 16000 one at 3263 ms (14x) on cf912d7, where the same commit
measures 3.7x here and the leg passed on the commit before. One timed
run of each let a busy moment on the runner decide the comparison: the
sizes are now compiled in turn three times and the best of each is
compared. A compile that is really quadratic is 13x at its best too.
@nicolas-maman
nicolas-maman marked this pull request as ready for review November 1, 2026 22:13
@nicolas-maman
nicolas-maman merged commit 48527bd into main Nov 1, 2026
46 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment