Implement high-level Lua APIs #2
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Extremely barebones implementation. Includes memory allocator,
high-level state wrapper, and basic string and table types. The core
idea of all types are going to be based around `LuaValue` (inspired by
mlua-rs).
`LuaValue` is an interface which all of the Lua type structs will
implement, and a consumer can leverage the Go type system to assert down
to or match against types. For example:
```go
import "github.com/CompeyDev/lei/lua"
state := lua.New()
table := state.CreateTable()
key, value := state.CreateString("hello"), state.CreateString("world")
table.Set(&key, &value)
roundtripValue := table.Get(&key)
// We can assert the type:
println(roundtripValue.(*lua.LuaString))
// If unsure about the type, we can switch for it:
switch roundtripValue.(type) {
case *lua.LuaString:
println(value.ToString())
case *lua.LuaTable:
println("The value was a table?")
default:
panic("Unexpected type, expected string!")
}
```
* Rewrote the allocator purely in Go and eliminated the C implementation for correctness. Also fixes limits not being respected due to improper decoding of the Go `MemoryState` struct in C * Memory state options (such as limits) must be specified before the creation of the Lua state, using the `NewWith` constructor. A `InitMemoryState` field was added to the options, and the `NewMemoryState` function was exported for this purpose * All unsafe methods which required or provided access to the underlying Lua state have been privated (`GetMemoryState`, `Lua.RawState`) * Removed redundant naming for `MemoryState` methods which included "memory" in the name itself * Introduced `StateWithMemory` holder struct which includes a `Pinner` that prevents GC or movement of the underlying `MemoryState` struct, which would otherwise cause memory corruption * Exported all unexported fields in `LuaOptions` and stopped storing the options in the `Lua` struct for no reason * The `Lua` struct is now only a thin wrapper around the unsafe Lua state * Renamed some private fields to be nicer and suit well with their methods. Also renamed the `luaState` method for the `LuaValue` interface to be `lua` instead * Added a cleanup finalizer for `Lua` which closes the unsafe Lua state and optionally triggers a GC in case `CollectGarbage` is set to true
We were defering a free of the bytecode too early. `luau_load` expects the bytecode to be held for as long as it is executing. Also handled ahe case for if the size of the bytecode is zero, for which case we send a NULL pointer to `luau_load` instead.
* Added `Compiler` struct as a safe abstraction around the Luau bytecode compiler. Returns structured `SyntaxError`s from `Compiler.Compile` * Allowed specifying a specific `Compiler` for `Lua` to use using state creation, and using `Lua.SetCompiler` * Added `Lua.Execute` to load and run a chunk of bytecode or source code and return its results, if any. Returns structured `LoadError`s
* Implement nil type with opaque handle to 0 * Add `deref` method to `LuaValue` interface, which supposed to be the index to type located stack after pushing it from the registry * Make `Lua.Execute` return `LuaValue`s * Make `Create`* functions in `Lua` return pointers (so that we can have finalizers that remove them from the registry in the future) * Implement `LuaTable.Iterable()` to convert a table into a iterable `map[LuaValue]LuaValue` * Implement `As[T]` to convert a `LuaValue` into a go type (currently supporting tables to structs or maps, nil, and strings)
* Annotations of the form `lua:"field_name"` can be used on struct fields to override the corresponding key name in Lua tables * If a field that is exported using PascalCase syntax cannot be found in the table, reflection falls back to trying to look for a corresponding camelCase name
* i.e., this allows for providing a nil pointer which is allowed as per the API * Now also accepts an empty string for providing no debug name * Remove hardcoded GCC headers include path
* Renamed `LoadError` to `LuaError` * Implement ability to construct Lua-callable functions from Go functions. This required implementing a registry on the Go side and a small wrapper around a cgo generated Go trampoline which received an opaque identifier which would be passed to the Go side, which would finally execute the required function * Fixed memory leak due to no cleanup finalizer for `LuaString` TODO: cleanup unused functions in the registry with no references TODO: allow safe function input for `Lua.CreateFunction` instead of expecting a `lua_CFunction`
* Refactored `functionRegistry.get` to return `functionEntry` which holds a reference to its parent registry and the ID of the created function * Pass this entry to the C-side wrapper for the dtor trampoline and have it pass it back to the Go-side dtor using `cgo.Handle`s for safety. Finally, the dtor cleans up the function referred to by the ID from the registry it belongs to
Instead of handing the unsafe raw `lua_State` as we did in the incomplete implementation, we now supply a "cooked" `Lua` state along with an array of all `LuaValue` arguments passed and instead of pushing the returns onto the stack and returning the count, we now expect an array of `LuaValue`s and an optional error.
Also corrected the `LuaValue` implementation for `LuaNil` to return `LUA_REFNIL` instead of 0 for `ref()` call.
Numbers are not an interned type, so we just store the value itself, avoiding any unneeded stack manipulation or registry storage.
* `LuaValue.lua()` method is now optional, may return a nil pointer * `LuaValue.deref()` is provided with a non-nil `*Lua` state pointer, since `LuaValue.lua()` may be nil if the type is stateless without an associated VM (currently only `LuaNumber` and `LuaNil`) * Added `LuaNumber` type corresponding to Go `float64` and value conversion implementations for it
Any panics within Go functions will be handled appropriately and returned as errors on the Lua side. All Lua related error throws are limited only to the C side, and we ensure that Go never calls any errors. By default, `LuaOptions.CatchPanics` is set to true. Also fixes our previous `lua_error` calls on the Go side which would cause a `longjmp` across the boundary causing a Go panic due to a violation of its stack winding rules.
This is a follow up inspired by some changes in 19f3c4f, which was the first commit to start using `go:generate` attributes, specific for cgo header generation. Instead of the `build` package being a CLI wrapper around the `go` command, compiling and injecting required shared libraries to link, we now have a thin command which compiles Luau CMake subprojects using Ninja. We refer to this command in `go:generate` attributes along with the required static libraries that the file expects. This means that users must run the following command to initialize the workspace: ``` go generate ./... ``` FUTURE: Might be worth not having a hard dependency on Ninja, or allowing users to specify their own CMake generator
* Implement `Lua.GetGlobal` and `Lua.SetGlobal` APIs * Implement `Lua.CreateUserData` and `LuaUserData` type, along with `IntoUserData` interface with a registry pattern to register metamethods, methods, and fields for custom userdata types * Use as shared `pushUpvalue` function to push registry patterned Go pointers to C, which must be passed back to Go using `cgo.Handle`s
Map iterations in Go are unordered. We keep track of all matches and order them in terms of their priority depending on the type of the match. This fixes the flaky test case for the tag overrides, as sometimes "name" field would match before the "user" field, which was incorrect behavior.
* Remove unused utility functions module * Provide direct bindings to `lua_open*` functions * Instead of hardcoding library names, refer to the exports in the header file instead * Validate the safety of libraries before loading them first
It might also be worth looking into supporting enabling and disabling the sandbox on the go, which `mlua` supports. It would, however, require a lot more involved approach. `mlua` is able to achieve this by maintaining a separate no-op thread which just holds references to Lua values and the state to prevent GC, along with maintaining a snapshot of the state before it is sandboxed. This way, they can simply `xpush` between the two threads holding their states in order to undo sandboxing, if required. If we consider this approach, it would require a pretty major refactor to also drop our dependence on the Lua registry, which we currently use to hold non-GCable references to data.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
leishould provide safe and validated high-level APIs to interact with Lua. The main source of entry is aLuastruct which can be used to construct various data types that can then be further handled individually.An important requirement is the ability to convert Lua values to and from Go values. We have a reflection module to do this in a terse and handy manner.
Implementation Requirements
vector3and)vector4thread,vectorbuffer