Repository navigation
fix: align call arity and native arg-presence semantics with Lua 5.4 - #32
Conversation
Four Lua-semantics bugs found by differential testing against Lua 5.4.9 across all three backends (TCC / GCC / interp): 1. Native functions used as values (local t = type, pcall(table.concat, t)) failed at runtime with a misleading message. semantic_analysis now reports it at compile time; math.pi / string.charpattern stay valid. 2. Optional arguments passed an explicit nil were rejected, though 5.4's luaL_optinteger/optlstring treat nil as absent. Fixed per function in table.concat/unpack, string.byte/sub/rep/find/match and utf8.codepoint/len/offset, keeping checkinteger params strict. 3. Argument count: Lua allows passing fewer or more arguments than the declared parameters (missing ones are nil, extras are discarded). Three gates rejected it: the compile-time check in CompileFunctioncall, the math-specialized direct call path, and the runtime check in CallByNameImpl. All three now pad with nil / discard extras, matching FlCallClosure which already behaved this way. The specialized direct call falls back to the normal path when the count differs - extra arguments previously leaked a raw TCC error into the user. 4. The interp backend treated every file-level local binding as a file-level function, so `local gv = f; gv(1)` failed with "function 'gv' not found". Only LocalFunction definitions may emit CALLNAME. Removing the runtime gate exposed that natives conflated "missing argument" with "explicit nil": 25 required-argument guards now raise bad argument (type/tostring/tonumber and the string/table/math/utf8 families). Contract change: FakeluaCallByName no longer throws on argument-count mismatch; CallByNameImpl fills every dispatched slot before the call, which is what the old Bug M guard was protecting against. Tests: new test/lua/basic/test_basic_call_arity.lua (expected 92, three backends) and the table/string/utf8 *_optnil suites; updated the three exception tests that asserted "too few arguments is a compile error" and the two runtime tests that asserted FakeluaCallByName throws. Full suite: 1584/1589 pass; the 5 failures are the known baseline (4 redis integration tests need a real server, test_websocket_echo_lua is flaky under load and passes standalone).
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #32 +/- ##
==========================================
+ Coverage 88.01% 88.07% +0.05%
==========================================
Files 124 124
Lines 25432 25486 +54
==========================================
+ Hits 22385 22446 +61
+ Misses 3047 3040 -7 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Removing the argument-count gate in CallByNameImpl left nothing between "the callee declares N parameters" and "write N slots into temp_arg_arr", which is a fixed 32-entry stack array. A callee declaring 33 parameters overflowed it: ASAN caught a 16-byte stack-buffer-overflow at vm.cpp:171 while running vm_cvar_call.parameter_limit_exceptions. Both marshalling sites now reject a declared parameter count above kMaxFunctionInputParams before touching the array, matching the guard FlCallClosure already had in src/state/fakelua.cpp:549. The CGen runtime header had the same latent write (c_runtime_header.h:452) and gets the same guard. Found by the PR #32 asan job.
|
Follow-up commit The asan job failed with a 16-byte stack-buffer-overflow at Cause: Bug 3 removed the argument-count gate in Fix: both marshalling sites now reject a declared parameter count above Good news for reviewers: |
Background
Differential testing against a locally built Lua 5.4.9 oracle (the system only ships 5.5), running each case on all three backends (TCC / GCC / interp), surfaced four systematic semantic divergences. This PR fixes them plus 25 required-argument guards that fell out of fix 3.
Oracle setup used for review reproduction: compile
lua-5.4.9/src/lualocally, then run the case files throughflua --jit_type=0|1|2 --entry mainand diff againstlua. New case files covering arity (35 cases), table-field/pcall (12), missing-argument (35) and iterator/callback (11) all report zero divergence after this PR.Bug 1 — native functions used as values
local t = type/pcall(table.concat, t)compiled and then failed at runtime with a misleading message.semantic_analysisnow rejects native functions used as values at compile time;math.pi/string.charpattern(field reads on native tables) stay valid.Bug 2 — explicit
nilfor optional argumentsOptional arguments rejected an explicit
nil, but Lua 5.4'sluaL_optinteger/luaL_optlstringtreatniland "absent" as equivalent. Fixed per-function intable.concat/unpack,string.byte/sub/rep/find/matchandutf8.codepoint/len/offset. Parameters that useluaL_checkintegerin real Lua remain strict (passingnilthere is still an error) — the two families are deliberately distinguished, so a per-function check against the 5.4 source is the useful review angle here.Bug 3 — argument count (the main fix)
Lua allows passing fewer or more arguments than the declared parameters: missing parameters are
nilinside the body, extra arguments are evaluated and discarded. Three independent gates rejected this:CompileFunctioncall(src/compile/c_gen.cpp) — "wrong number of arguments" is no longer a compile error. Padding/truncation is centralized inBuildLocalFunctionCall.TryCompileSpecDirectCall) — now verifies the count and falls back to the normal path when it differs. Extra arguments previously leaked a raw TCC compiler error to the user.CallByNameImpl(src/jit/vm.cpp) — removed, so the existing pad/discard loop takes effect.The three call paths (CGen direct call,
FlCallClosureclosure-variable call,FakeluaCallByNamedispatch) are now consistent —FlCallClosurealready behaved correctly, which is what the differential tests were calibrated against.Contract change (needs reviewer attention)
FakeluaCallByNameno longer throws on argument-count mismatch.CallByNameImplfills every dispatched slot before the call, which is exactly what the old "Bug M" guard was protecting against — so no uninitialized slot is reachable. Three tests intest/test_exception.cppthat asserted "too few arguments is a compile error" and two intest/test_runtime.cppthat assertedFakeluaCallByNamethrows were rewritten to the new contract (now: compiles, then throwsattempt to perform arithmetic on non-numeric valueatCalltime because the missing parameter isnil).Follow-on: 25 required-argument guards
Removing gate 3 exposed that natives conflated "missing argument" with "explicit nil" —
type()/tostring()silently succeeded instead of raising "bad argument". 25 guards now raise:type/tostring/tonumber, thestring.*,table.*families,math.max/math.min, andutf8.{codepoint,codes,len,offset}. Defaults that are genuinely optional (table.create,loadfile,dofile) were left alone.Bug 4 — interp treats file-level locals as functions
src/interp/codegen.cppclassified every file-levellocalbinding withdefining_func == nullptras a file-level function, solocal gv = f; gv(1)emittedCALLNAMEand failed at runtime withfunction 'gv' not found. OnlyLocalFunctiondefinition nodes may emitCALLNAME; everything else goes through the closure path (which is what the C backends already do).Tests
test/lua/basic/test_basic_call_arity.lua(expected 92, calibrated against 5.4.9, run on all three backends) covering all six call paths: direct call with fewer/more args, closure-variable call, file-level function call (incl. math-specialized with extra args), multi-value expansion, vararg, and native call with extra args.*_optnilsuites for table (75), string (1146), utf8 (691).test_syntax_treecases for Bug 1.test_redis.integration_*need a live Redis server, andtest_net.test_websocket_echo_luais flaky under load (passes standalone in ~1s).Review focus
src/compile/c_gen.cpp— the three arity gates and the spec-call fallback condition.src/jit/vm.cpp— removal of the runtime throw; confirm every dispatched slot is written before the call.src/interp/codegen.cpp— thefile_level_funcnarrowing.src/native/**— the 25 guards; spot-check a few against the Lua 5.4.9 source forcheck*vsopt*.