feat(plugins): Lua plugin + hooks system#1752
Open
suhaanthayyil wants to merge 12 commits into
Open
Conversation
Introduce a pure-Go embedded Lua runtime (github.com/yuin/gopher-lua) so third parties can author no-build-step plugins that subscribe to lifecycle hooks and contribute commands, layered on the existing kubectl-style binary plugins. Phase 1 (foundation): - Sandboxed LState builder: curated stdlib (base/string/table/math only), os/io/package/debug stripped, escape-hatch base globals removed, per-run context timeout via SetContext. - plugin.json manifest types with strict (DisallowUnknownFields) parsing and validation of name, hooks, commands, and capabilities. - Loader/registry that discovers plugins from the per-user managed dir (~/.local/share/entire/plugins/lua/<name>) and repo-local .entire/plugins, gated by an explicit allow-list; repo-local plugins never auto-run. - settings.PluginSettings allow-list with per-plugin capability grants (http/exec/fs/net), strict validation, and team/local merge — mirroring the external_agents opt-in pattern. - Observer hook bus POC: entire.on/entire.log, panic isolation, per-hook timeout. Empty-hook dispatch measured at ~1.5us/op (CGO_ENABLED=0). Builds with CGO_ENABLED=0 on all targets; gopher-lua is MIT-licensed. Co-authored-by: Cursor <cursoragent@cursor.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR introduces the foundation for a pure-Go embedded Lua plugin runtime (via github.com/yuin/gopher-lua) that can load allow-listed plugins from user-managed and repo-local directories, parse strict plugin.json manifests, and dispatch observer-style hook callbacks inside a sandboxed Lua state. It extends the existing settings model to include an explicit per-plugin allow-list with capability grants.
Changes:
- Add
gopher-luadependency and a newcmd/entire/cli/pluginspackage implementing sandboxing, manifest parsing/validation, discovery, and observer hook dispatch. - Extend settings schema with
pluginsallow-list entries + strict capability-name validation, plus merge semantics across settings layers. - Add unit tests/benchmarks covering sandboxing, settings parsing/validation, manifest parsing, and registry discovery/dispatch behavior.
Reviewed changes
Copilot reviewed 16 out of 17 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| go.mod | Adds github.com/yuin/gopher-lua dependency. |
| go.sum | Adds checksums for gopher-lua. |
| cmd/entire/cli/settings/settings.go | Adds plugins allow-list + capability definitions/validation and merge behavior. |
| cmd/entire/cli/settings/settings_plugins_test.go | Tests plugin allow-list parsing and validation errors. |
| cmd/entire/cli/plugins/sandbox.go | Implements a curated/sandboxed Lua state with stripped globals and context-based abort. |
| cmd/entire/cli/plugins/sandbox_test.go | Tests curated libs, removed escape hatches, and timeout abort behavior; includes benchmark. |
| cmd/entire/cli/plugins/registry.go | Implements plugin discovery + observer hook dispatch over loaded plugins. |
| cmd/entire/cli/plugins/registry_test.go | Tests discovery rules (allow-list gating, repo-local opt-in, collision precedence) and dispatch semantics. |
| cmd/entire/cli/plugins/plugin.go | Defines LoadedPlugin and loads a plugin (manifest + sandbox + entry script). |
| cmd/entire/cli/plugins/paths.go | Defines managed/user/repo plugin directory resolution and per-plugin data dir pathing. |
| cmd/entire/cli/plugins/names.go | Duplicates/exports plugin-name validation rules used by the binary plugin dispatcher. |
| cmd/entire/cli/plugins/marshal.go | Adds Go→Lua marshaling for JSON-like payload shapes. |
| cmd/entire/cli/plugins/manifest.go | Adds strict plugin.json parsing with structural validation (hooks/commands/capabilities). |
| cmd/entire/cli/plugins/manifest_test.go | Tests strict manifest parsing and validation failures. |
| cmd/entire/cli/plugins/hooks.go | Defines the stable hook-name surface and known-hook validation helpers. |
| cmd/entire/cli/plugins/config.go | Defines execution timeouts for plugin load and observer callback dispatch. |
| cmd/entire/cli/plugins/api.go | Installs the entire Lua API surface (on, logging) and rebinds print() to internal logging. |
Comment on lines
+38
to
+48
| top := l.GetTop() | ||
| msg := "" | ||
| var msgSb40 strings.Builder | ||
| for i := 1; i <= top; i++ { | ||
| if i > 1 { | ||
| msgSb40.WriteString("\t") | ||
| } | ||
| msgSb40.WriteString(l.ToStringMeta(l.Get(i)).String()) | ||
| } | ||
| msg += msgSb40.String() | ||
| p.log(slog.LevelInfo, msg) |
Comment on lines
+3
to
+20
| import ( | ||
| "context" | ||
| "log/slog" | ||
| "os" | ||
| "path/filepath" | ||
| "sort" | ||
|
|
||
| "github.com/entireio/cli/cmd/entire/cli/logging" | ||
| "github.com/entireio/cli/cmd/entire/cli/settings" | ||
| lua "github.com/yuin/gopher-lua" | ||
| ) | ||
|
|
||
| // Registry holds the loaded, allow-listed plugins for a process and dispatches | ||
| // hook events to them. All access to plugin Lua states goes through the | ||
| // registry, which serializes it (LStates are not goroutine-safe). | ||
| type Registry struct { | ||
| plugins []*LoadedPlugin | ||
| } |
Comment on lines
+114
to
+116
| func (r *Registry) Add(p *LoadedPlugin) { | ||
| r.plugins = append(r.plugins, p) | ||
| } |
Comment on lines
+142
to
+152
| func (r *Registry) FireObserver(ctx context.Context, hook string, payload map[string]any) { | ||
| if r == nil || len(r.plugins) == 0 { | ||
| return | ||
| } | ||
| logCtx := logging.WithComponent(ctx, "plugins") | ||
| for _, p := range r.plugins { | ||
| for _, cb := range p.callbacks[hook] { | ||
| r.invokeObserver(ctx, logCtx, p, hook, cb, payload) | ||
| } | ||
| } | ||
| } |
Comment on lines
+179
to
+184
| func (r *Registry) Close() { | ||
| for _, p := range r.plugins { | ||
| p.Close() | ||
| } | ||
| r.plugins = nil | ||
| } |
Comment on lines
+118
to
+136
| // Plugins returns the loaded plugins in load order. | ||
| func (r *Registry) Plugins() []*LoadedPlugin { | ||
| return r.plugins | ||
| } | ||
|
|
||
| // Len returns the number of loaded plugins. | ||
| func (r *Registry) Len() int { | ||
| return len(r.plugins) | ||
| } | ||
|
|
||
| // PluginNames returns the sorted names of loaded plugins. | ||
| func (r *Registry) PluginNames() []string { | ||
| names := make([]string, 0, len(r.plugins)) | ||
| for _, p := range r.plugins { | ||
| names = append(names, p.Manifest.Name) | ||
| } | ||
| sort.Strings(names) | ||
| return names | ||
| } |
Phase 2 (observer hook bus):
- Process-wide plugin registry (plugins.FireHook) lazily built from settings +
worktree root; fast no-op when no plugin is enabled, serialized dispatch,
per-hook timeout and panic isolation so a misbehaving observer can never
break a lifecycle event or git hook.
- Wire seams: DispatchLifecycleEvent maps agent.EventType to hook names and
fires after successful handling; checkpoint_saved fires post-SaveStep in
lifecycle.go; post_commit fires in strategy PostCommit; pre_push (observer)
fires in strategy PrePush.
- Lua API: entire.on, entire.log.{debug,info,warn,error}, entire.kv (durable
per-plugin JSON store in the data dir), read-only accessors
(plugin_name/version/source/repo_root/data_dir), and print routed to the log.
- Capability-gated stubs for entire.http/exec/fs/net: calling one without the
matching grant raises a Lua error naming the missing capability; with the
grant it reports "not implemented yet" (real impls land in phase 3). Denial
is authoritative now so plugins fail loudly rather than silently no-op.
- Hook payloads expose operational metadata (ids, agent, model, file paths)
but never prompt text or file contents.
- Tests: kv round-trip + persistence, capability gating, accessors, lifecycle
hook mapping/payload.
Co-authored-by: Cursor <cursoragent@cursor.com>
Phase 3 (capabilities + hardening):
- Real, capability-gated implementations replacing the phase-2 stubs:
- entire.http.get/post: http/https only, per-call timeout, 5 MiB response cap.
- entire.exec.run: subprocess with timeout, returns {stdout, stderr, code}.
- entire.fs.read/write: confined to the repo root and plugin data dir;
traversal outside either is rejected. read capped at 10 MiB.
- entire.net.connect: gated but intentionally unimplemented (http is the
supported network path); granting net still cannot open raw sockets.
- Every privileged call raises a Lua error naming the missing capability when
ungranted, so plugins fail loud rather than silently no-op.
- Hardening: process-wide kill switch (ENTIRE_PLUGINS_DISABLED), configurable
per-hook timeout (ENTIRE_PLUGIN_HOOK_TIMEOUT_MS), and capability calls bounded
by their own timeout derived from the hook context. Deliberately avoid
gopher-lua SetMx (it os.Exit(3)s the whole process on global memory).
- Trust boundary documented: the allow-list plus capability grants are the
boundary; the sandbox limits accidental damage, not a deliberately malicious
allow-listed plugin. Repo-local plugins still never auto-run.
- Tests: exec/fs/http real behavior, fs traversal rejection, http scheme
rejection, capability denial, net stub, kill switch.
Co-authored-by: Cursor <cursoragent@cursor.com>
Phase 4 (mutating hooks): - prepare_commit_msg: a plugin granted the new commit_msg capability may return a trailer string that is appended to the commit message. Plugin trailers land AFTER the built-in Entire-Checkpoint trailer (never displacing it) and before the git comment block; multiple plugins contribute in deterministic load order. Wired via a deferred append in strategy PrepareCommitMsg so it runs on the normal and amend paths but not on merge/squash/sequence operations. - pre_push veto: a plugin granted the new pre_push capability may return false (with an optional reason) to veto a push. The veto aborts the user's push via a non-zero pre-push hook exit. It runs before the built-in OPF rewrite and checkpoint-ref push so a veto short-circuits that work. Non-capable plugins still receive the pre_push observer fire; their return value is ignored. - New capabilities commit_msg and pre_push added to the settings allow-list and validation; the first veto in load order supplies the reported reason. - Ordering and multi-plugin semantics documented in mutate.go. - Tests: trailer contribution/gating/ordering, veto with/without capability, observer-still-runs, and commit-message trailer insertion. Co-authored-by: Cursor <cursoragent@cursor.com>
Phase 5 (plugin commands):
- entire.command{name, short, run} registers a subcommand invocable as
`entire <name>`; the run callback receives a Lua array of the remaining args
and may return an integer exit code (nil means 0). entire.print/entire.write
give commands real stdout output (distinct from the log-routed global print).
- Dispatch (MaybeRunLuaCommand) enforces resolution order built-in > Lua
command > entire-<name> binary: it is called from main.go before the binary
dispatcher, and both defer to built-ins. Plugins are only loaded when the
first arg is a plugin-shaped, non-built-in name, so built-ins pay no cost.
- Commands run under the cancellable process context (not a short hook timeout)
so interactive/long-running commands work; panics and errors map to exit 1.
- Registry.Commands/FindCommand expose contributed commands for listing/help;
first plugin in load order wins a name collision.
- Tests: command execution/args/exit codes, error and default-zero paths,
listing/precedence, and built-in-wins dispatch resolution.
Co-authored-by: Cursor <cursoragent@cursor.com>
Phase 6 (distribution): - `entire plugin install <git-url|path>` now routes by source: a git URL is cloned into the managed lua/<name> dir (keyed by plugin.json name), a directory with a plugin.json is copied in as a Lua plugin, and an entire-<name> file keeps the existing binary-plugin behavior. --ref pins a git tag, branch, or commit. - `entire plugin update [name]` fast-forwards git-installed Lua plugins (a pinned branch/tag updates; a pinned commit stays put); with no name it updates all git-installed Lua plugins. Install provenance is recorded in a small .entire-install.json for reproducible updates. - `entire plugin list` now shows Lua plugins (with source/ref) alongside binary plugins; `entire plugin remove <name>` removes either kind by bare name. - Install only places files: a Lua plugin stays inert until allow-listed and enabled in settings, so installing an untrusted URL cannot run its code. The install output reminds the user of the opt-in step. - Tests: git-URL detection, local-path install/list/force/remove, and a real git clone + update round-trip. Co-authored-by: Cursor <cursoragent@cursor.com>
Phase 7 (docs + SDK + examples): - docs/architecture/plugins-lua.md: full reference — trust model, layout, manifest, settings allow-list, the entire.* API, hooks (observer + mutating), command resolution order, distribution, and implementation map. - examples/plugins/entire.lua: EmmyLua/lua-language-server type-annotation stub for editor autocompletion of the plugin API. - examples/plugins/checkpoint-notify: observer hooks + kv + a command + optional exec; examples/plugins/models-updater: a command using http + fs. Plus a README for the examples dir. - CLAUDE.md (and AGENTS.md via symlink): plugins package pointer and the key invariants; docs/security-and-privacy.md: third-party plugin trust section. - Test that loads the example plugins so they stay valid in CI. Co-authored-by: Cursor <cursoragent@cursor.com>
Update the plugin command group's help to cover both binary and Lua plugins, the allow-list opt-in requirement, the managed parent-dir precedence, and the new install/update surface. Co-authored-by: Cursor <cursoragent@cursor.com>
Repo-local .entire/plugins/<name> plugins could be activated by a committed team .entire/settings.json entry (enabled + capabilities), so a malicious PR could ship both the plugin and its enable and have the code run on clone — contradicting the documented trust model. Repo-local plugins are now governed solely by the per-developer, uncommitted .entire/settings.local.json: settings.LocalPluginGrants reads that file only, and grantForSource requires a SourceRepo plugin's grant to come from it. User-global installed plugins are unchanged and still resolve from the merged team+local allow-list. Docs and the plugins invariant note are updated to match.
The url and ref for `entire plugin install` were passed to `git clone` and `git checkout` without validation or a `--` separator, so a value beginning with `-` (e.g. `--upload-pack=<cmd>`) could be parsed as a git option and lead to command execution. Reject url/ref values starting with `-`, pass `--` before the clone's positional arguments, and re-validate the stored ref on update.
Contributor
|
Thanks for putting this together. Could you add a bit more context on what’s driving the need for a new plugin system and what problem it’s meant to solve? This feels like a bigger design decision than we should make in a PR alone, especially since it could affect more than just this implementation. I think it would be useful to discuss it with a wider group first, maybe through an RFD that lays out the use cases, requirements, and alternatives. We also already have plugin mechanisms for external agents and CLI commands, both using the git/kubectl-style approach of finding plugins through $PATH. Depending on the goal here, for example, responding to events, we might be able to build on the existing CLI plugin system by adding hooks similar to the external agent ones, rather than introducing a third mechanism. My main concern is making sure we agree on the problem and the overall approach before getting too far into the implementation. |
|
my 2 cents (feel free to disregard—this isn't my project after all):
|
…/update + staleness plugin Turn the models-updater stub into a genuinely useful Lua plugin that keeps .entire/models.json fresh and makes drift visible, exercising the whole plugin surface: command + observer hook + http + fs + kv + capability-gating. - `entire models-update`: http.get the LiteLLM pricing JSON (URL overridable), fs.read the cached copy, diff per-model input/output rates, fs.write the merged result, and kv-record the refresh. Local-only model ids (present in the cache but absent upstream) are preserved verbatim, never erased. - `--check`: fetch + report drift only, no write, non-zero exit on drift (CI). - `session_start` hook: bumps a durable logical session counter and emits one gentle log-level nudge only when the cache is stale (never fetched, or not refreshed in 25 sessions); quiet on the fresh path. Handles two sandbox realities without inventing host APIs: the curated stdlib exposes no wall clock (staleness is measured in sessions via kv, not days) and no JSON library, and tonumber rejects exponent notation — so the plugin ships a tiny string-aware object scanner plus a mantissa/exponent number parser, with a relative-tolerance rate compare so notation differences never read as drift. Add behavior tests that stub entire.http with an httptest server and assert fs/kv side effects and exit codes: first-fetch write, drift + local-only preservation, --check (drift/clean), HTTP error, realistic JSON shapes, and the session_start nudge (never-fetched / stale / fresh). Register the session_start hook in the example-load test and the manifest.
…dater to use it
The Lua plugin sandbox opens only base/string/table/math, so plugins that
handle JSON had to hand-roll a scanner. Add entire.json.decode/encode as an
always-available API (pure and deterministic — it reaches nothing outside the
Lua state, so it is not capability-gated), registered alongside
entire.log/kv/print.
- decode(str): json.Unmarshal into interface{} then the shared toLuaValue
conversion — objects->tables, arrays->sequences, numbers->Lua numbers
(fractional and scientific notation such as 3e-06 preserved via float64),
true/false/null->boolean/boolean/nil. Raises on invalid JSON.
- encode(value): the inverse conversion (new fromLuaValue) then json.Marshal —
a dense 1..n table encodes as an array, any other table (incl. empty) as an
object. A depth guard rejects cyclic/over-deep tables; values with no JSON
form (functions, userdata) raise.
Rewrite the models-updater example to decode resp.body in one call, dropping
its hand-rolled top-level object scanner and custom mantissa/exponent number
parser (~130 net lines). Rates are now read by exact key (the decoy
input_cost_per_token_above_128k_tokens field is naturally ignored) and
scientific-notation rates parse correctly for free, so equal values in
different notations no longer false-drift. Preserved local-only model ids are
re-encoded with entire.json.encode. Behavior (--check/--write/drift-diff,
local-only preservation, kv refresh marker, session_start staleness nudge) is
unchanged.
Document entire.json in the plugins-lua architecture doc and the entire.lua
type-annotation stub. Add focused decode/encode round-trip tests; the nine
models-updater behavior tests pass unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a pure-Go embedded Lua runtime (
github.com/yuin/gopher-lua) so thirdparties can author no-build-step plugins that subscribe to lifecycle/git hooks
and contribute commands, layered on top of the existing kubectl-style
entire-<name>binary plugins without breaking them.Grown phase by phase; each phase is a separate conventional commit.
Design constraints (all met)
CGO_ENABLED=0everywhere — gopher-lua only (MIT, in.allowed-licenses). VerifiedCGO_ENABLED=0 go build ./...and the releasebinary builds clean.
plugin.jsonparsed strictly (DisallowUnknownFields), no TOML..entire/plugins/) never auto-run — require an explicitallow-list entry, mirroring the
external_agentsopt-in posture.http/exec/fs/netplus the mutating-hook grantscommit_msg/pre_pushare denied unless granted per-plugin; ungranted callsfail loud.
Decision gates
the "few ms" gate. The no-plugins-enabled fast path does zero filesystem work.
a blocker.
Trust model
Opt-in at every step: inert until allow-listed, repo-local never auto-runs,
privileged APIs capability-gated,
installonly places files, andENTIRE_PLUGINS_DISABLED=1is a process-wide kill switch. The allow-list +capability grants are the trust boundary. See
docs/architecture/plugins-lua.md.Phases
stdlib, escape hatches stripped, context timeout),
plugin.jsonmanifest,loader/registry (user + repo dirs),
settings.pluginsallow-list withcapability grants, observer hook-bus POC, unit tests.
DispatchLifecycleEvent,post-
SaveStep(checkpoint_saved),PostCommit, andPrePush; event→Luatable payloads;
entire.on/entire.log/entire.kv, read-only accessors;capability-gated stubs for
http/exec/fs/net.http/exec/fs(repo-confined) with timeouts + size caps; kill switch; configurable hooktimeout; documented trust boundary.
prepare_commit_msgtrailer (after thebuilt-in
Entire-Checkpointtrailer) andpre_pushveto → non-zero exit,both capability-gated, with documented ordering vs OPF and multi-plugin
semantics.
entire.command{...}+entire.print;resolution order built-in > Lua command >
entire-<name>binary.entire plugin install <git-url|path>(
--refpin),entire plugin update, list/remove surfacing Lua + binaryplugins.
docs/architecture/plugins-lua.md,CLAUDE.md/AGENTS.md + security-and-privacy.md updates,
examples/plugins/entire.luaLSP stub, and two example plugins.
Test plan
CGO_ENABLED=0 go test ./cmd/entire/cli/ ./cmd/entire/cli/settings/... ./cmd/entire/cli/plugins/...(2677 pass) and./cmd/entire/cli/strategy/...(814 pass)CGO_ENABLED=0 go build ./...and release binary buildgolangci-lintclean; dependency license check passesNever merge — long-lived branch.