WIP: Lua Intellisense

[Nickelony]

This pull request introduces significant improvements to Lua intellisense support, refactors and extends configuration path management, and makes various usability and reliability enhancements throughout the codebase. The most notable changes are the integration of a new Lua intellisense provider, the addition of a document manager for intellisense, updates to configuration directory structure, and improved error handling and user experience in the scripting studio.

Lua Intellisense Integration and Infrastructure:

• Added a new ILuaIntellisenseProvider integration in the LuaStudio class, including event hooks for diagnostics and semantic token updates, and implemented navigation to definitions and live diagnostics in editors. (TombIDE/TombIDE.ScriptingStudio/LuaStudio.Intellisense.cs, TombIDE/TombIDE.ScriptingStudio/LuaStudio.cs) [1] [2] [3] [4] [5] [6]
• Introduced LuaIntellisenseDocumentManager to manage document states, diagnostics, and semantic tokens for Lua files, supporting synchronization and versioning for language server communication. (TombIDE/TombIDE.ScriptingStudio/Services/LuaIntellisense/LuaIntellisenseDocumentManager.cs)

Configuration and Path Management Enhancements:

• Improved fallback logic for the default program directory to use AppContext.BaseDirectory if the assembly location is unavailable. (GlobalPaths.cs)
• Refactored and expanded configuration paths: added TextEditorThemesDirectory and renamed LuaColorConfigsDirectory to LuaThemeConfigsDirectory for clearer separation of themes and color schemes. (GlobalPaths.cs)

Usability and Error Handling Improvements:

• Enhanced error handling in script generation by logging exceptions for better debugging. (TombIDE/TombIDE.ScriptingStudio/LuaStudio.cs)
• Ensured language files are opened with the correct editor type and improved logic for applying user settings and diagnostics when switching tabs or updating settings. (TombIDE/TombIDE.ScriptingStudio/LuaStudio.cs) [1] [2] [3] [4]

Documentation Updates:

• Updated ExternalResources.md to acknowledge the use of Codicons icon geometry from the VSCode Codicons project under the MIT license. (ExternalResources.md)

[Copilot]

Pull request overview

This PR introduces groundwork for Lua intellisense support (via Lua language server) and modernizes editor diagnostics/tooltips/rendering to support richer hover/diagnostic UX across scripting editors.

Changes:

• Migrates multiple projects from a local AvalonEdit DLL reference to a NuGet AvalonEdit package reference (and adds MdXaml where needed).
• Replaces legacy line-based error storage with a shared TextEditorDiagnostic model + severity, updates renderers/tooltips to use diagnostics and adds markdown tooltip rendering.
• Adds Lua language server integration (completion, hover, signature help, diagnostics, and go-to-definition via F12/CTRL+Click) plus UI command + localization plumbing.

Reviewed changes

Copilot reviewed 49 out of 50 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
TombLib/TombLib.Test/TombLib.Test.csproj	Switches AvalonEdit dependency to NuGet package for test project.
TombLib/TombLib.Scripting/Workers/ErrorDetectionWorker.cs	Simplifies worker argument passing and returns typed diagnostics.
TombLib/TombLib.Scripting/TombLib.Scripting.csproj	Moves AvalonEdit to NuGet and adds MdXaml dependency.
TombLib/TombLib.Scripting/Rendering/TextRendering.cs	Removes legacy bitmap-based zig-zag underline rendering helper.
TombLib/TombLib.Scripting/Rendering/MarkdownToolTipRenderer.cs	Adds markdown-to-WPF tooltip rendering with code block handling + highlighting.
TombLib/TombLib.Scripting/Rendering/ErrorRenderer.cs	Updates underlining to render by diagnostics severity (wavy for errors, straight for others).
TombLib/TombLib.Scripting/Rendering/BookmarkRenderer.cs	Renders bookmark highlights using new bookmark collection logic.
TombLib/TombLib.Scripting/Objects/TextEditorDiagnosticSeverity.cs	Introduces diagnostic severity enum + label helper.
TombLib/TombLib.Scripting/Objects/TextEditorDiagnostic.cs	Introduces diagnostic model with offsets and range helpers.
TombLib/TombLib.Scripting/Interfaces/IErrorDetector.cs	Changes error detector API to return IReadOnlyList<TextEditorDiagnostic>.
TombLib/TombLib.Scripting/Bases/TextEditorBase.cs	Adds diagnostic storage/tooltips (incl. markdown), popup-based hover UI, and anchor-based bookmark tracking.
TombLib/TombLib.Scripting.Tomb1Main/Utils/ErrorDetector.cs	Ports TR1 error detection to emit TextEditorDiagnostic with offsets.
TombLib/TombLib.Scripting.Tomb1Main/TombLib.Scripting.Tomb1Main.csproj	Switches AvalonEdit dependency to NuGet package.
TombLib/TombLib.Scripting.Tomb1Main/Tomb1MainEditor.cs	Uses diagnostics + markdown hover tooltips in editor integration.
TombLib/TombLib.Scripting.Tomb1Main/Services/Implementations/GameflowHoverService.cs	Formats hover output as markdown inline code for improved tooltip rendering.
TombLib/TombLib.Scripting.Lua/TombLib.Scripting.Lua.csproj	Switches AvalonEdit dependency to NuGet package.
TombLib/TombLib.Scripting.Lua/Services/ILuaIntellisenseProvider.cs	Adds intellisense provider abstraction for Lua LSP features + diagnostics.
TombLib/TombLib.Scripting.Lua/Objects/LuaSignatureInfo.cs	Adds signature help DTOs (signature + parameters).
TombLib/TombLib.Scripting.Lua/Objects/LuaHoverInfo.cs	Adds hover DTO with markdown/plaintext flag.
TombLib/TombLib.Scripting.Lua/Objects/LuaDefinitionLocation.cs	Adds definition location DTO for navigation.
TombLib/TombLib.Scripting.Lua/Objects/LuaCompletionItem.cs	Adds completion item DTO.
TombLib/TombLib.Scripting.Lua/Objects/LuaCompletionData.cs	Bridges completion DTOs into AvalonEdit ICompletionData.
TombLib/TombLib.Scripting.Lua/LuaEditor.SignatureHelp.cs	Adds signature help popup UI rendering for Lua editor.
TombLib/TombLib.Scripting.Lua/LuaEditor.Navigation.cs	Adds go-to-definition via F12 and CTRL+Click navigation request flow.
TombLib/TombLib.Scripting.Lua/LuaEditor.Intellisense.cs	Adds completion/hover/signature help plumbing + transient tooltip dismissal logic.
TombLib/TombLib.Scripting.Lua/LuaEditor.cs	Makes LuaEditor partial, wires intellisense + enables live diagnostics underlining.
TombLib/TombLib.Scripting.GameFlowScript/TombLib.Scripting.GameFlowScript.csproj	Switches AvalonEdit dependency to NuGet package.
TombLib/TombLib.Scripting.ClassicScript/Utils/ErrorDetector.cs	Ports classic script error detection to TextEditorDiagnostic with offsets.
TombLib/TombLib.Scripting.ClassicScript/TombLib.Scripting.ClassicScript.csproj	Switches AvalonEdit dependency to NuGet package.
TombLib/TombLib.Scripting.ClassicScript/ClassicScriptEditor.cs	Updates editor to apply diagnostics and suppress definition hovers when diagnostics present.
TombIDE/TombIDE.Shared/Resources/Localization/PL/TombIDE.xml	Adds localized label for “Go to Definition”.
TombIDE/TombIDE.Shared/Resources/Localization/EN/TombIDE.xml	Adds localized label for “Go to Definition”.
TombIDE/TombIDE.Shared/Resources/Localization/Localization.cs	Adds GoToDefinition localization property.
TombIDE/TombIDE.Shared/IDEConfiguration.cs	Adds persisted config value for Lua language server path.
TombIDE/TombIDE.ScriptingStudio/UI/UIKeys.cs	Adds F12 shortcut mapping for “Go to Definition”.
TombIDE/TombIDE.ScriptingStudio/UI/UICommand.cs	Adds GoToDefinition command enum value.
TombIDE/TombIDE.ScriptingStudio/UI/DocumentModePresets/ContextMenus/Lua.xml	Adds context menu item for go-to-definition in Lua editor.
TombIDE/TombIDE.ScriptingStudio/TombIDE.ScriptingStudio.csproj	Switches AvalonEdit dependency to NuGet package.
TombIDE/TombIDE.ScriptingStudio/Settings/LuaSettingsControl.Designer.cs	Enables and labels autocomplete setting checkbox.
TombIDE/TombIDE.ScriptingStudio/Settings/ClassicScriptSettingsControl.cs	Updates settings preview to clear diagnostics instead of legacy errors.
TombIDE/TombIDE.ScriptingStudio/Services/LuaIntellisense/LuaLanguageServerSettingsFactory.cs	Creates Lua LS settings payload (runtime/workspace/completion/telemetry).
TombIDE/TombIDE.ScriptingStudio/Services/LuaIntellisense/LuaLanguageServerResponseParser.cs	Parses LSP responses for completion/hover/definition/signature help.
TombIDE/TombIDE.ScriptingStudio/Services/LuaIntellisense/LuaLanguageServerPathHelper.cs	Normalizes paths and URIs for file-based LSP communication.
TombIDE/TombIDE.ScriptingStudio/Services/LuaIntellisense/LuaLanguageServerLocator.cs	Resolves LS executable from config/PATH/VS Code extensions and persists discovered path.
TombIDE/TombIDE.ScriptingStudio/Services/LuaIntellisense/LuaLanguageServerIntellisenseProvider.cs	Implements ILuaIntellisenseProvider atop the LSP client with document sync + diagnostics publishing.
TombIDE/TombIDE.ScriptingStudio/Services/LuaIntellisense/LuaLanguageServerDiagnosticsParser.cs	Converts published LSP diagnostics into TextEditorDiagnostic offsets/messages.
TombIDE/TombIDE.ScriptingStudio/Services/LuaIntellisense/LuaLanguageServerClient.cs	Implements JSON-RPC/LSP process client (start/init, request/notify, read loop, config handling).
TombIDE/TombIDE.ScriptingStudio/Services/LuaIntellisense/LuaIntellisenseDocumentManager.cs	Tracks document open/change state and associates diagnostics with versions.
TombIDE/TombIDE.ScriptingStudio/LuaStudio.Intellisense.cs	Wires Lua studio tabs to LS provider (sync documents, apply diagnostics, handle definition navigation).
TombIDE/TombIDE.ScriptingStudio/LuaStudio.cs	Initializes/disposes Lua intellisense and routes GoToDefinition command to the active Lua editor.

Files not reviewed (1)

• TombIDE/TombIDE.ScriptingStudio/Settings/LuaSettingsControl.Designer.cs: Language not supported

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 65 out of 67 changed files in this pull request and generated 2 comments.

Files not reviewed (2)

• TombIDE/TombIDE.ScriptingStudio/Settings/LuaSettingsControl.Designer.cs: Language not supported
• TombLib/TombLib.Forms/Forms/FormAbout.designer.cs: Language not supported

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 67 out of 69 changed files in this pull request and generated 3 comments.

Files not reviewed (2)

• TombIDE/TombIDE.ScriptingStudio/Settings/LuaSettingsControl.Designer.cs: Language not supported
• TombLib/TombLib.Forms/Forms/FormAbout.designer.cs: Language not supported

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 102 out of 105 changed files in this pull request and generated 3 comments.

Files not reviewed (3)

• TombIDE/TombIDE.ScriptingStudio/Settings/LuaSettingsControl.Designer.cs: Language not supported
• TombLib/TombLib.Forms/Forms/FormAbout.designer.cs: Language not supported
• TombLib/TombLib.Scripting.Lua/LuaTextBox.Designer.cs: Language not supported

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 104 out of 107 changed files in this pull request and generated 2 comments.

Files not reviewed (3)

• TombIDE/TombIDE.ScriptingStudio/Settings/LuaSettingsControl.Designer.cs: Language not supported
• TombLib/TombLib.Forms/Forms/FormAbout.designer.cs: Language not supported
• TombLib/TombLib.Scripting.Lua/LuaTextBox.Designer.cs: Language not supported

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Nickelony]

@copilot Could you please review this PR and check for any regressions? Any improvements we could make? Any de-duplications possible? Anything which can be simplified? Is anything overengineered? Please rate the changes and whether any additional work needs to be done.

[Copilot]

@copilot Could you please review this PR and check for any regressions? Any improvements we could make? Any de-duplications possible? Anything which can be simplified? Is anything over...

I've done a thorough review of all 108 changed files (~10K lines added). Here's my assessment:

Overall Rating: 8/10 — Well-architected, solid implementation

The feature introduces a proper LSP-based Lua intellisense system with good separation of concerns. The architecture is clean and the code quality is high overall.

Fixes Applied (commit 06097ec, 4f58ed9)

1. BrushHelpers.cs — Added input validation for ColorConverter.ConvertFromString(). Previously would crash on invalid color strings with an unhelpful InvalidCastException.
2. LuaCompletionData.cs — Clarified confusing null-conditional logic (_resolveTask?.IsCanceled != false || _resolveTask.IsFaulted → explicit is null || .IsCanceled || .IsFaulted). The old logic worked but was hard to reason about.
3. LuaThemeRepository.cs — Empty catch {} now logs the file path and error message for theme loading failures.
4. LuaLanguageServerResponseParser.cs — Replaced magic number priority weights (1000000, 20000, 10000, 9000, 7000, -5000) with named constants and used the existing LuaLanguageServerCompletionKind enum instead of raw ints.
5. LuaLanguageServerPathHelper.cs — NormalizeLocalPath(Uri) now reuses NormalizeLocalPath(string) instead of duplicating the Replace('/') + GetFullPath() logic.

What's Working Well

• Partial class organization of LuaEditor is excellent — clean separation by feature (Completion, Hover, Navigation, SignatureHelp, SemanticHighlighting).
• LuaIntellisenseDocumentManager is well-designed with proper locking and immutable document snapshots.
• ErrorRenderer correctly caches all pens as static readonly frozen instances.
• ReadHeadersAsync uses proper buffered reading with chunk-based buffer management.
• EnsureSemanticTokensColorizerAttached correctly prevents duplicate registrations with Contains() check.
• LuaLineParser has comprehensive test coverage (18 interaction rule tests + 4 parser tests + 9 theme tests = 31 total).
• TextMate infrastructure (Colorizer, DocumentLineList, ThemeStyleResolver) follows standard adapter patterns.

Potential Improvements (not blocking)

1. LuaLineParser state machine duplication — IsInsideCommentOrString(), StripLineComment(), and EnumerateStructuralCharacters() each contain nearly identical state transition logic for LongString/LongComment/SingleQuote/DoubleQuote. A shared iteration helper with callbacks could reduce this ~160 lines of near-duplicate code.

2. Data classes as records — LuaDefinitionLocation, LuaHoverInfo, LuaParameterInfo are immutable sealed classes with only read-only properties. They're perfect candidates for sealed record which gives free value equality and more concise syntax.

3. MarkdownToolTipRenderer (656 lines) — functional but dense. Could benefit from extracting the code-block rendering logic into a small helper class.

4. LuaTheme.Normalize() — 13 repetitive NormalizeValue() calls. Works fine, but could be more maintainable with a reflection-based approach or property map.

5. Fire-and-forget tasks in LuaLanguageServerIntellisenseProvider — ObserveBackgroundTask() logs failures via Debug output, which is sufficient for development but could be missed in production. Consider NLog for these.

Not Overengineered

• The LSP client implementation is appropriately complex for what it does. The buffered I/O, request/response correlation, and process lifecycle management are all necessary.
• The TextMate highlighting infrastructure is standard adapter code.
• The completion priority system is reasonable — the weights create a sensible ordering.

No Regressions Found

• CommentOutLines/UncommentLines correctly preserves blank lines.
• Event handlers are properly unsubscribed in LuaEditor_Unloaded.
• CancellationToken handling follows correct patterns.
• The SelectionLength - 1 in comment methods is correct (strips trailing newline from AppendLine).