Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@

## Table of Contents

- [What's new (2026-06-19) — Plugin SDK](#whats-new-2026-06-19--plugin-sdk)
- [What's new (2026-06-19) — MCP Structured Output](#whats-new-2026-06-19--mcp-structured-output)
- [What's new (2026-06-19) — Tweened Drag](#whats-new-2026-06-19--tweened-drag)
- [What's new (2026-06-19) — Process-Doc (SOP) Generator](#whats-new-2026-06-19--process-doc-sop-generator)
Expand Down Expand Up @@ -83,6 +84,12 @@

---

## What's new (2026-06-19) — Plugin SDK

Third-party `AC_*` commands via entry points. Full reference: [`docs/source/Eng/doc/new_features/v31_features_doc.rst`](docs/source/Eng/doc/new_features/v31_features_doc.rst).

- **`discover_plugins` / `load_plugins`** (`AC_list_plugins` / `AC_load_plugins`, `ac_*`): a pip package registers new executor commands declaratively in the `je_auto_control.commands` entry-point group; AutoControl discovers and registers them at runtime (immediately usable from JSON flows, socket server, scheduler, MCP). Broken plugins are skipped; the declarative, namespaced complement to the runtime path loader.

## What's new (2026-06-19) — MCP Structured Output

MCP 2025-06-18 structured tool output. Full reference: [`docs/source/Eng/doc/new_features/v30_features_doc.rst`](docs/source/Eng/doc/new_features/v30_features_doc.rst).
Expand Down
7 changes: 7 additions & 0 deletions README/README_zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@

## 目录

- [本次更新 (2026-06-19) — Plugin SDK](#本次更新-2026-06-19--plugin-sdk)
- [本次更新 (2026-06-19) — MCP 结构化输出](#本次更新-2026-06-19--mcp-结构化输出)
- [本次更新 (2026-06-19) — 缓动拖拽](#本次更新-2026-06-19--缓动拖拽)
- [本次更新 (2026-06-19) — 流程文档(SOP)生成器](#本次更新-2026-06-19--流程文档sop生成器)
Expand Down Expand Up @@ -82,6 +83,12 @@

---

## 本次更新 (2026-06-19) — Plugin SDK

通过 entry points 注册第三方 `AC_*` 指令。完整参考:[`docs/source/Zh/doc/new_features/v31_features_doc.rst`](../docs/source/Zh/doc/new_features/v31_features_doc.rst)。

- **`discover_plugins` / `load_plugins`**(`AC_list_plugins` / `AC_load_plugins`、`ac_*`):pip 包以 `je_auto_control.commands` entry-point 组声明式注册新执行器指令;AutoControl 于运行期发现并注册(立即可用于 JSON 流程、socket server、调度器、MCP)。坏插件会跳过;为运行期路径加载器的声明式、带命名空间对应物。

## 本次更新 (2026-06-19) — MCP 结构化输出

MCP 2025-06-18 结构化工具输出。完整参考:[`docs/source/Zh/doc/new_features/v30_features_doc.rst`](../docs/source/Zh/doc/new_features/v30_features_doc.rst)。
Expand Down
7 changes: 7 additions & 0 deletions README/README_zh-TW.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@

## 目錄

- [本次更新 (2026-06-19) — Plugin SDK](#本次更新-2026-06-19--plugin-sdk)
- [本次更新 (2026-06-19) — MCP 結構化輸出](#本次更新-2026-06-19--mcp-結構化輸出)
- [本次更新 (2026-06-19) — 緩動拖曳](#本次更新-2026-06-19--緩動拖曳)
- [本次更新 (2026-06-19) — 流程文件(SOP)產生器](#本次更新-2026-06-19--流程文件sop產生器)
Expand Down Expand Up @@ -82,6 +83,12 @@

---

## 本次更新 (2026-06-19) — Plugin SDK

透過 entry points 註冊第三方 `AC_*` 指令。完整參考:[`docs/source/Zh/doc/new_features/v31_features_doc.rst`](../docs/source/Zh/doc/new_features/v31_features_doc.rst)。

- **`discover_plugins` / `load_plugins`**(`AC_list_plugins` / `AC_load_plugins`、`ac_*`):pip 套件以 `je_auto_control.commands` entry-point 群組宣告式註冊新執行器指令;AutoControl 於執行期探索並註冊(立即可用於 JSON 流程、socket server、排程器、MCP)。壞外掛會略過;為執行期路徑載入器的宣告式、具命名空間對應物。

## 本次更新 (2026-06-19) — MCP 結構化輸出

MCP 2025-06-18 結構化工具輸出。完整參考:[`docs/source/Zh/doc/new_features/v30_features_doc.rst`](../docs/source/Zh/doc/new_features/v30_features_doc.rst)。
Expand Down
48 changes: 48 additions & 0 deletions docs/source/Eng/doc/new_features/v31_features_doc.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
==================================================
New Features (2026-06-19) — Plugin SDK
==================================================

A third-party pip package can now register new ``AC_*`` executor commands
declaratively via a setuptools **entry point** in the
``je_auto_control.commands`` group — turning the monolith into an ecosystem
(how pytest/Playwright grew). AutoControl discovers them at runtime;
discovered commands are immediately usable from JSON action files, the
socket server, the scheduler, and MCP. Pure standard library
(``importlib.metadata``); full stack.

.. contents::
:local:
:depth: 2


Authoring a plugin
=================

A plugin package exposes an entry point whose target is a factory returning
a ``{command_name: handler}`` mapping::

# in the plugin's pyproject.toml
[project.entry-points."je_auto_control.commands"]
my_pack = "my_pack.commands:provide"

# my_pack/commands.py
def provide():
return {"AC_my_command": lambda **kw: {"ok": True}}


Discovering & loading
====================

::

from je_auto_control import discover_plugins, load_plugins

discover_plugins() # {command_name: handler} from all plugins
load_plugins() # discover + register into the executor

Broken plugins are skipped (logged), not fatal. Exposed as
``AC_list_plugins`` (discover names) / ``AC_load_plugins`` (discover +
register) and ``ac_list_plugins`` / ``ac_load_plugins``. The entry-point
source is injectable, so discovery is unit-testable without installing a
real plugin. This is the declarative, namespaced complement to the existing
runtime path loader.
1 change: 1 addition & 0 deletions docs/source/Eng/eng_index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ Comprehensive guides for all AutoControl features.
doc/new_features/v28_features_doc
doc/new_features/v29_features_doc
doc/new_features/v30_features_doc
doc/new_features/v31_features_doc
doc/ocr_backends/ocr_backends_doc
doc/observability/observability_doc
doc/operations_layer/operations_layer_doc
Expand Down
44 changes: 44 additions & 0 deletions docs/source/Zh/doc/new_features/v31_features_doc.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
==========================================
新功能 (2026-06-19) — Plugin SDK
==========================================

第三方 pip 套件現在可透過 setuptools **entry point**(``je_auto_control.commands``
群組)宣告式地註冊新的 ``AC_*`` 執行器指令——把單體變成生態系(pytest /
Playwright 的成長方式)。AutoControl 於執行期探索它們;探索到的指令立即可
用於 JSON action 檔、socket server、排程器與 MCP。純標準庫
(``importlib.metadata``);走完整五層。

.. contents::
:local:
:depth: 2


撰寫外掛
========

外掛套件提供一個 entry point,其目標為回傳 ``{command_name: handler}``
對應的工廠函式::

# 外掛的 pyproject.toml
[project.entry-points."je_auto_control.commands"]
my_pack = "my_pack.commands:provide"

# my_pack/commands.py
def provide():
return {"AC_my_command": lambda **kw: {"ok": True}}


探索與載入
==========

::

from je_auto_control import discover_plugins, load_plugins

discover_plugins() # 來自所有外掛的 {command_name: handler}
load_plugins() # 探索 + 註冊到執行器

壞掉的外掛會被略過(記錄),不致命。對應 ``AC_list_plugins``(探索名稱)
/ ``AC_load_plugins``(探索 + 註冊)以及 ``ac_list_plugins`` /
``ac_load_plugins``。entry-point 來源可注入,因此探索能在不安裝真實外掛
的情況下單元測試。這是既有執行期路徑載入器的宣告式、具命名空間的對應物。
1 change: 1 addition & 0 deletions docs/source/Zh/zh_index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ AutoControl 所有功能的完整使用指南。
doc/new_features/v28_features_doc
doc/new_features/v29_features_doc
doc/new_features/v30_features_doc
doc/new_features/v31_features_doc
doc/ocr_backends/ocr_backends_doc
doc/observability/observability_doc
doc/operations_layer/operations_layer_doc
Expand Down
5 changes: 5 additions & 0 deletions je_auto_control/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -199,6 +199,10 @@
from je_auto_control.utils.tween_drag import (
easing_names, tween_drag, tween_points,
)
# Plugin SDK: discover/load third-party AC_* commands via entry points
from je_auto_control.utils.plugin_sdk import (
COMMANDS_GROUP, discover_plugins, load_plugins,
)
# Background popup/interrupt watchdog (unattended automation)
from je_auto_control.utils.watchdog import (
PopupWatchdog, WatchdogRule, default_popup_watchdog,
Expand Down Expand Up @@ -633,6 +637,7 @@ def start_autocontrol_gui(*args, **kwargs):
"analyze_heal_log", "heal_stats", "scan_secrets",
"describe_step", "generate_sop", "write_sop",
"easing_names", "tween_drag", "tween_points",
"COMMANDS_GROUP", "discover_plugins", "load_plugins",
# MCP server
"AuditLogger", "HttpMCPServer", "MCPContent", "MCPPrompt",
"MCPPromptArgument", "MCPResource", "MCPServer", "MCPTool",
Expand Down
12 changes: 12 additions & 0 deletions je_auto_control/gui/script_builder/command_schema.py
Original file line number Diff line number Diff line change
Expand Up @@ -681,6 +681,18 @@ def _add_misc_specs(specs: List[CommandSpec]) -> None:
description="Drag along an eased path; 'start'/'end' [x,y] via JSON "
"view.",
))
specs.append(CommandSpec(
"AC_list_plugins", "Tools", "List Plugin Commands",
fields=(FieldSpec("group", FieldType.STRING, optional=True,
default="je_auto_control.commands"),),
description="Discover third-party AC_* commands from entry points.",
))
specs.append(CommandSpec(
"AC_load_plugins", "Tools", "Load Plugin Commands",
fields=(FieldSpec("group", FieldType.STRING, optional=True,
default="je_auto_control.commands"),),
description="Discover + register third-party plugin commands.",
))
specs.append(CommandSpec(
"AC_generate_sop", "Report", "Generate SOP Document",
fields=(
Expand Down
14 changes: 14 additions & 0 deletions je_auto_control/utils/executor/action_executor.py
Original file line number Diff line number Diff line change
Expand Up @@ -2882,6 +2882,18 @@ def _tween_drag(start: List[int], end: List[int], steps: int = 30,
return {"points": result["points"]}


def _list_plugins(group: str = "je_auto_control.commands") -> Dict[str, Any]:
"""Adapter: discover third-party plugin command names (no register)."""
from je_auto_control.utils.plugin_sdk import discover_plugins
return {"commands": sorted(discover_plugins(group))}


def _load_plugins(group: str = "je_auto_control.commands") -> Dict[str, Any]:
"""Adapter: discover + register third-party plugin commands."""
from je_auto_control.utils.plugin_sdk import load_plugins
return {"loaded": load_plugins(group)}


class Executor:
"""
Executor
Expand Down Expand Up @@ -3111,6 +3123,8 @@ def __init__(self):
"AC_scan_secrets": _scan_secrets,
"AC_generate_sop": _generate_sop,
"AC_tween_drag": _tween_drag,
"AC_list_plugins": _list_plugins,
"AC_load_plugins": _load_plugins,
"AC_a11y_record_start": _a11y_record_start,
"AC_a11y_record_stop": _a11y_record_stop,
"AC_a11y_record_events": _a11y_record_events,
Expand Down
25 changes: 24 additions & 1 deletion je_auto_control/utils/mcp_server/tools/_factories.py
Original file line number Diff line number Diff line change
Expand Up @@ -2550,6 +2550,29 @@ def tween_drag_tools() -> List[MCPTool]:
]


def plugin_sdk_tools() -> List[MCPTool]:
_G = {"group": {"type": "string"}}
return [
MCPTool(
name="ac_list_plugins",
description=("Discover third-party AC_* commands registered via "
"the 'je_auto_control.commands' entry-point group "
"(without registering them). Returns {commands}."),
input_schema=schema(dict(_G)),
handler=h.list_plugins,
annotations=READ_ONLY,
),
MCPTool(
name="ac_load_plugins",
description=("Discover and register third-party plugin commands "
"into the executor. Returns {loaded} names."),
input_schema=schema(dict(_G)),
handler=h.load_plugins,
annotations=SIDE_EFFECT_ONLY,
),
]


def unattended_tools() -> List[MCPTool]:
return [
MCPTool(
Expand Down Expand Up @@ -3607,7 +3630,7 @@ def media_assert_tools() -> List[MCPTool]:
checkpoint_tools, set_of_marks_tools, screen_state_tools,
input_macro_tools, resilience_tools,
ci_annotation_tools, clipboard_history_tools, audit_analysis_tools,
process_doc_tools, tween_drag_tools,
process_doc_tools, tween_drag_tools, plugin_sdk_tools,
screen_record_tools,
process_and_shell_tools, remote_desktop_tools, gamepad_tools,
usb_passthrough_tools, assertion_tools, data_source_tools,
Expand Down
10 changes: 10 additions & 0 deletions je_auto_control/utils/mcp_server/tools/_handlers.py
Original file line number Diff line number Diff line change
Expand Up @@ -1236,6 +1236,16 @@ def tween_drag(start, end, steps=30, easing="ease_in_out_quad",
easing=easing, button=button)["points"]}


def list_plugins(group="je_auto_control.commands"):
from je_auto_control.utils.plugin_sdk import discover_plugins
return {"commands": sorted(discover_plugins(group))}


def load_plugins(group="je_auto_control.commands"):
from je_auto_control.utils.plugin_sdk import load_plugins as _load
return {"loaded": _load(group)}


def vlm_locate(description: str,
screen_region: Optional[List[int]] = None,
model: Optional[str] = None) -> Optional[List[int]]:
Expand Down
6 changes: 6 additions & 0 deletions je_auto_control/utils/plugin_sdk/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
"""Plugin SDK: discover & load third-party AC_* commands via entry points."""
from je_auto_control.utils.plugin_sdk.plugin_sdk import (
COMMANDS_GROUP, discover_plugins, load_plugins,
)

__all__ = ["COMMANDS_GROUP", "discover_plugins", "load_plugins"]
62 changes: 62 additions & 0 deletions je_auto_control/utils/plugin_sdk/plugin_sdk.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
"""Plugin SDK — discover and load third-party ``AC_*`` commands.

Turns the monolith into an ecosystem: a third-party pip package can register
new executor commands declaratively via a setuptools **entry point** in the
``je_auto_control.commands`` group, and AutoControl discovers them at runtime
(no path hacking, namespaced, install-time discoverable — distinct from the
runtime path loader). Each entry point loads to a *factory* returning a
``{command_name: callable}`` mapping, which is registered into the executor.

Pure standard library (``importlib.metadata``); imports no ``PySide6``. The
entry-point source is injectable, so discovery is unit-testable without
installing a real plugin.
"""
from typing import Any, Callable, Dict, List, Optional

COMMANDS_GROUP = "je_auto_control.commands"


def _entry_points(group: str) -> List[Any]:
from importlib import metadata
return list(metadata.entry_points(group=group))


def discover_plugins(group: str = COMMANDS_GROUP,
entry_points: Optional[List[Any]] = None
) -> Dict[str, Callable[..., Any]]:
"""Return ``{command_name: handler}`` from every plugin entry point.

Each entry point loads to a callable factory returning a command
mapping; broken plugins are skipped (logged), not fatal. Pass
``entry_points`` to inject a fake source in tests.
"""
points = entry_points if entry_points is not None else _entry_points(group)
commands: Dict[str, Callable[..., Any]] = {}
for point in points:
try:
factory = point.load()
produced = factory()
except (ImportError, AttributeError, TypeError, ValueError) as error:
from je_auto_control.utils.logging.logging_instance import (
autocontrol_logger)
autocontrol_logger.warning(
"plugin entry point %r failed: %r",
getattr(point, "name", point), error)
continue
if isinstance(produced, dict):
commands.update(produced)
return commands


def load_plugins(group: str = COMMANDS_GROUP,
entry_points: Optional[List[Any]] = None) -> List[str]:
"""Discover plugin commands and register them into the executor.

Returns the sorted names of the commands that were registered.
"""
commands = discover_plugins(group, entry_points)
if commands:
from je_auto_control.utils.executor.action_executor import (
add_command_to_executor)
add_command_to_executor(commands)
return sorted(commands)
Loading
Loading