Skip to content

fix: align call arity and native arg-presence semantics with Lua 5.4 - #32

Merged
esrrhs merged 5 commits into
masterfrom
fix/call-arity-lua-semantics
Oct 15, 2026
Merged

esrrhs merged 5 commits into
masterfrom
fix/call-arity-lua-semantics

Conversation

@esrrhs

@esrrhs esrrhs commented Oct 14, 2026

Copy link
Copy Markdown
Owner

Background

Differential testing against a locally built Lua 5.4.9 oracle (the system only ships 5.5), running each case on all three backends (TCC / GCC / interp), surfaced four systematic semantic divergences. This PR fixes them plus 25 required-argument guards that fell out of fix 3.

Oracle setup used for review reproduction: compile lua-5.4.9/src/lua locally, then run the case files through flua --jit_type=0|1|2 --entry main and diff against lua. New case files covering arity (35 cases), table-field/pcall (12), missing-argument (35) and iterator/callback (11) all report zero divergence after this PR.

Bug 1 — native functions used as values

local t = type / pcall(table.concat, t) compiled and then failed at runtime with a misleading message. semantic_analysis now rejects native functions used as values at compile time; math.pi / string.charpattern (field reads on native tables) stay valid.

Bug 2 — explicit nil for optional arguments

Optional arguments rejected an explicit nil, but Lua 5.4's luaL_optinteger / luaL_optlstring treat nil and "absent" as equivalent. Fixed per-function in table.concat/unpack, string.byte/sub/rep/find/match and utf8.codepoint/len/offset. Parameters that use luaL_checkinteger in real Lua remain strict (passing nil there is still an error) — the two families are deliberately distinguished, so a per-function check against the 5.4 source is the useful review angle here.

Bug 3 — argument count (the main fix)

Lua allows passing fewer or more arguments than the declared parameters: missing parameters are nil inside the body, extra arguments are evaluated and discarded. Three independent gates rejected this:

  1. Compile-time check in CompileFunctioncall (src/compile/c_gen.cpp) — "wrong number of arguments" is no longer a compile error. Padding/truncation is centralized in BuildLocalFunctionCall.
  2. Math-specialized direct call (TryCompileSpecDirectCall) — now verifies the count and falls back to the normal path when it differs. Extra arguments previously leaked a raw TCC compiler error to the user.
  3. Runtime check in CallByNameImpl (src/jit/vm.cpp) — removed, so the existing pad/discard loop takes effect.

The three call paths (CGen direct call, FlCallClosure closure-variable call, FakeluaCallByName dispatch) are now consistent — FlCallClosure already behaved correctly, which is what the differential tests were calibrated against.

Contract change (needs reviewer attention)

FakeluaCallByName no longer throws on argument-count mismatch. CallByNameImpl fills every dispatched slot before the call, which is exactly what the old "Bug M" guard was protecting against — so no uninitialized slot is reachable. Three tests in test/test_exception.cpp that asserted "too few arguments is a compile error" and two in test/test_runtime.cpp that asserted FakeluaCallByName throws were rewritten to the new contract (now: compiles, then throws attempt to perform arithmetic on non-numeric value at Call time because the missing parameter is nil).

Follow-on: 25 required-argument guards

Removing gate 3 exposed that natives conflated "missing argument" with "explicit nil" — type() / tostring() silently succeeded instead of raising "bad argument". 25 guards now raise: type/tostring/tonumber, the string.*, table.* families, math.max/math.min, and utf8.{codepoint,codes,len,offset}. Defaults that are genuinely optional (table.create, loadfile, dofile) were left alone.

Bug 4 — interp treats file-level locals as functions

src/interp/codegen.cpp classified every file-level local binding with defining_func == nullptr as a file-level function, so local gv = f; gv(1) emitted CALLNAME and failed at runtime with function 'gv' not found. Only LocalFunction definition nodes may emit CALLNAME; everything else goes through the closure path (which is what the C backends already do).

Tests

  • New test/lua/basic/test_basic_call_arity.lua (expected 92, calibrated against 5.4.9, run on all three backends) covering all six call paths: direct call with fewer/more args, closure-variable call, file-level function call (incl. math-specialized with extra args), multi-value expansion, vararg, and native call with extra args.
  • New *_optnil suites for table (75), string (1146), utf8 (691).
  • New test_syntax_tree cases for Bug 1.
  • Full suite: 1584/1589 pass. The 5 failures are the known baseline: 4 test_redis.integration_* need a live Redis server, and test_net.test_websocket_echo_lua is flaky under load (passes standalone in ~1s).

Review focus

  • src/compile/c_gen.cpp — the three arity gates and the spec-call fallback condition.
  • src/jit/vm.cpp — removal of the runtime throw; confirm every dispatched slot is written before the call.
  • src/interp/codegen.cpp — the file_level_func narrowing.
  • src/native/** — the 25 guards; spot-check a few against the Lua 5.4.9 source for check* vs opt*.

Four Lua-semantics bugs found by differential testing against Lua 5.4.9
across all three backends (TCC / GCC / interp):

1. Native functions used as values (local t = type, pcall(table.concat, t))
   failed at runtime with a misleading message. semantic_analysis now
   reports it at compile time; math.pi / string.charpattern stay valid.

2. Optional arguments passed an explicit nil were rejected, though 5.4's
   luaL_optinteger/optlstring treat nil as absent. Fixed per function in
   table.concat/unpack, string.byte/sub/rep/find/match and
   utf8.codepoint/len/offset, keeping checkinteger params strict.

3. Argument count: Lua allows passing fewer or more arguments than the
   declared parameters (missing ones are nil, extras are discarded).
   Three gates rejected it: the compile-time check in CompileFunctioncall,
   the math-specialized direct call path, and the runtime check in
   CallByNameImpl. All three now pad with nil / discard extras, matching
   FlCallClosure which already behaved this way. The specialized direct
   call falls back to the normal path when the count differs - extra
   arguments previously leaked a raw TCC error into the user.

4. The interp backend treated every file-level local binding as a file-level
   function, so `local gv = f; gv(1)` failed with "function 'gv' not found".
   Only LocalFunction definitions may emit CALLNAME.

Removing the runtime gate exposed that natives conflated "missing argument"
with "explicit nil": 25 required-argument guards now raise bad argument
(type/tostring/tonumber and the string/table/math/utf8 families).

Contract change: FakeluaCallByName no longer throws on argument-count
mismatch; CallByNameImpl fills every dispatched slot before the call, which
is what the old Bug M guard was protecting against.

Tests: new test/lua/basic/test_basic_call_arity.lua (expected 92, three
backends) and the table/string/utf8 *_optnil suites; updated the three
exception tests that asserted "too few arguments is a compile error" and
the two runtime tests that asserted FakeluaCallByName throws.

Full suite: 1584/1589 pass; the 5 failures are the known baseline (4 redis
integration tests need a real server, test_websocket_echo_lua is flaky under
load and passes standalone).
@codecov-commenter

codecov-commenter commented Oct 15, 2026 •

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.07%. Comparing base (4b345a3) to head (dd19597).
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##           master      #32      +/-   ##
==========================================
+ Coverage   88.01%   88.07%   +0.05%     
==========================================
  Files         124      124              
  Lines       25432    25486      +54     
==========================================
+ Hits        22385    22446      +61     
+ Misses       3047     3040       -7     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Removing the argument-count gate in CallByNameImpl left nothing between
"the callee declares N parameters" and "write N slots into temp_arg_arr",
which is a fixed 32-entry stack array. A callee declaring 33 parameters
overflowed it: ASAN caught a 16-byte stack-buffer-overflow at vm.cpp:171
while running vm_cvar_call.parameter_limit_exceptions.

Both marshalling sites now reject a declared parameter count above
kMaxFunctionInputParams before touching the array, matching the guard
FlCallClosure already had in src/state/fakelua.cpp:549. The CGen runtime
header had the same latent write (c_runtime_header.h:452) and gets the
same guard.

Found by the PR #32 asan job.
@esrrhs

esrrhs commented Oct 15, 2026

Copy link
Copy Markdown
Owner Author

Follow-up commit 28bbe1c — ASAN caught a real bug this PR introduced.

The asan job failed with a 16-byte stack-buffer-overflow at src/jit/vm.cpp:171 inside CallByNameImpl, while running vm_cvar_call.parameter_limit_exceptions (that test registers a callee declaring 33 parameters, i.e. above kMaxFunctionInputParams = 32, and expects a throw).

Cause: Bug 3 removed the argument-count gate in CallByNameImpl, which was also the only thing standing between "the callee declares N parameters" and "write N slots into temp_arg_arr" — a fixed 32-entry stack array. Lua's pad-with-nil semantics are right, but the array still has a hard ceiling.

Fix: both marshalling sites now reject a declared parameter count above kMaxFunctionInputParams before touching the array — CallByNameImpl (src/jit/vm.cpp) and the CGen runtime header's FlCallClosure (src/compile/c_runtime_header.h:452, same latent write). This mirrors the guard FlCallClosure already had at src/state/fakelua.cpp:549.

Good news for reviewers: fuzz (fuzz_compile) and fuzz (fuzz_differential) both passed, and debug/release/cov/macos builds were green before this commit.

@esrrhs
esrrhs merged commit 4e5aef0 into master Oct 15, 2026
9 checks passed
@esrrhs
esrrhs deleted the fix/call-arity-lua-semantics branch October 15, 2026 06:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants