Skip to content

feat: add cross-platform desktop automation tool (macOS + Windows)#43

Open
NSIETeam wants to merge 1 commit into
OrionStarAI:opensourcefrom
NSIETeam:feat/desktop-automation-tool
Open

feat: add cross-platform desktop automation tool (macOS + Windows)#43
NSIETeam wants to merge 1 commit into
OrionStarAI:opensourcefrom
NSIETeam:feat/desktop-automation-tool

Conversation

@NSIETeam

Copy link
Copy Markdown

What

Adds a new desktop_automation tool with 17 cross-platform actions for macOS and Windows.

Actions

Category Actions
App control launch_app, quit_app, wait_for_app
Window management window_manager (minimize, maximize, restore, close, front, tile_left/right/top/bottom, fullscreen)
Keyboard keyboard, type_text, hotkey
Mouse mouse, drag, scroll
System screenshot, clipboard, screen_info, get_active_app, list_windows, get_window_position
Scripting run_script (AppleScript on macOS, PowerShell on Windows)

Implementation

  • macOS: osascript + cliclick for keyboard/mouse, no extra dependencies beyond brew install cliclick
  • Windows: Win32 API via PowerShell P/Invoke (zero external dependencies)
  • Safety: dangerous actions (run_script, quit_app, drag) show confirmation warning; YOLO mode skips confirmation
  • Validation: parameter validation with clear error messages

Origin

Ported from Otto project. Adapted for EasyCode:

  • Removed ProcessGuard dependency, uses standard child_process.exec
  • Removed DoctorService dependency, inline which cliclick check
  • Same API surface and behavior

Testing

  • TypeScript compiles without errors
  • Unit tests (planned: parameter validation, macOS/Windows path routing)
  • Integration tests on macOS (requires cliclick)
  • Integration tests on Windows

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant