diff --git a/README.md b/README.md index 9f286ef5..eef5ad11 100644 --- a/README.md +++ b/README.md @@ -13,6 +13,7 @@ ## Table of Contents +- [What's new (2026-06-19) — Tweened Drag](#whats-new-2026-06-19--tweened-drag) - [What's new (2026-06-19) — Process-Doc (SOP) Generator](#whats-new-2026-06-19--process-doc-sop-generator) - [What's new (2026-06-19) — Heal Analytics & Secret Scan](#whats-new-2026-06-19--heal-analytics--secret-scan) - [What's new (2026-06-19) — CI Annotations & Clipboard History](#whats-new-2026-06-19--ci-annotations--clipboard-history) @@ -81,6 +82,12 @@ --- +## What's new (2026-06-19) — Tweened Drag + +Deterministic eased drags. Full reference: [`docs/source/Eng/doc/new_features/v29_features_doc.rst`](docs/source/Eng/doc/new_features/v29_features_doc.rst). + +- **`tween_points` / `tween_drag` / `easing_names`** (`AC_tween_drag`, `ac_tween_drag`): drag from `start` to `end` along an eased curve (linear / ease_in_out_quad / ease_out_cubic / ease_in_cubic) — deterministic, pure-math path, injectable sink for tests; complements the humanized jitter. + ## What's new (2026-06-19) — Process-Doc (SOP) Generator Turn an action list into a step-by-step SOP. Full reference: [`docs/source/Eng/doc/new_features/v28_features_doc.rst`](docs/source/Eng/doc/new_features/v28_features_doc.rst). diff --git a/README/README_zh-CN.md b/README/README_zh-CN.md index 9d5eb4dc..3532ed2d 100644 --- a/README/README_zh-CN.md +++ b/README/README_zh-CN.md @@ -12,6 +12,7 @@ ## 目录 +- [本次更新 (2026-06-19) — 缓动拖拽](#本次更新-2026-06-19--缓动拖拽) - [本次更新 (2026-06-19) — 流程文档(SOP)生成器](#本次更新-2026-06-19--流程文档sop生成器) - [本次更新 (2026-06-19) — 修复分析与机密扫描](#本次更新-2026-06-19--修复分析与机密扫描) - [本次更新 (2026-06-19) — CI 注解与剪贴板历史](#本次更新-2026-06-19--ci-注解与剪贴板历史) @@ -80,6 +81,12 @@ --- +## 本次更新 (2026-06-19) — 缓动拖拽 + +确定性的缓动拖拽。完整参考:[`docs/source/Zh/doc/new_features/v29_features_doc.rst`](../docs/source/Zh/doc/new_features/v29_features_doc.rst)。 + +- **`tween_points` / `tween_drag` / `easing_names`**(`AC_tween_drag`、`ac_tween_drag`):沿缓动曲线从 `start` 拖到 `end`(linear / ease_in_out_quad / ease_out_cubic / ease_in_cubic)——确定性、纯数学路径、测试可注入 sink;补足人性化抖动。 + ## 本次更新 (2026-06-19) — 流程文档(SOP)生成器 把动作列表转成逐步 SOP。完整参考:[`docs/source/Zh/doc/new_features/v28_features_doc.rst`](../docs/source/Zh/doc/new_features/v28_features_doc.rst)。 diff --git a/README/README_zh-TW.md b/README/README_zh-TW.md index 8113b876..10163151 100644 --- a/README/README_zh-TW.md +++ b/README/README_zh-TW.md @@ -12,6 +12,7 @@ ## 目錄 +- [本次更新 (2026-06-19) — 緩動拖曳](#本次更新-2026-06-19--緩動拖曳) - [本次更新 (2026-06-19) — 流程文件(SOP)產生器](#本次更新-2026-06-19--流程文件sop產生器) - [本次更新 (2026-06-19) — 修復分析與機密掃描](#本次更新-2026-06-19--修復分析與機密掃描) - [本次更新 (2026-06-19) — CI 註解與剪貼簿歷史](#本次更新-2026-06-19--ci-註解與剪貼簿歷史) @@ -80,6 +81,12 @@ --- +## 本次更新 (2026-06-19) — 緩動拖曳 + +決定性的緩動拖曳。完整參考:[`docs/source/Zh/doc/new_features/v29_features_doc.rst`](../docs/source/Zh/doc/new_features/v29_features_doc.rst)。 + +- **`tween_points` / `tween_drag` / `easing_names`**(`AC_tween_drag`、`ac_tween_drag`):沿緩動曲線從 `start` 拖到 `end`(linear / ease_in_out_quad / ease_out_cubic / ease_in_cubic)——決定性、純數學路徑、測試可注入 sink;補足人性化抖動。 + ## 本次更新 (2026-06-19) — 流程文件(SOP)產生器 把動作清單轉成逐步 SOP。完整參考:[`docs/source/Zh/doc/new_features/v28_features_doc.rst`](../docs/source/Zh/doc/new_features/v28_features_doc.rst)。 diff --git a/docs/source/Eng/doc/new_features/v29_features_doc.rst b/docs/source/Eng/doc/new_features/v29_features_doc.rst new file mode 100644 index 00000000..cc6b319d --- /dev/null +++ b/docs/source/Eng/doc/new_features/v29_features_doc.rst @@ -0,0 +1,29 @@ +================================================== +New Features (2026-06-19) — Tweened Drag +================================================== + +Deterministic eased drags along a curved path (PyAutoGUI-style ``tween``), +complementing the existing humanized jitter. Pure standard library; full +stack. The point math is pure and unit-testable; dispatch goes through an +injectable sink. + +.. contents:: + :local: + :depth: 2 + + +Usage +===== + +:: + + from je_auto_control import tween_points, tween_drag, easing_names + + tween_points((0, 0), (100, 50), steps=20, easing="ease_out_cubic") + tween_drag((0, 0), (300, 200), steps=40, easing="ease_in_out_quad") + +``tween_points`` returns ``steps + 1`` eased points between two +coordinates; ``tween_drag`` presses at the start, moves through the points, +and releases at the end. Easings: ``linear`` / ``ease_in_out_quad`` / +``ease_out_cubic`` / ``ease_in_cubic`` (see :func:`easing_names`). Exposed +as ``AC_tween_drag`` / ``ac_tween_drag`` (``start`` / ``end`` as ``[x, y]``). diff --git a/docs/source/Eng/eng_index.rst b/docs/source/Eng/eng_index.rst index 00be320e..93fc81a6 100644 --- a/docs/source/Eng/eng_index.rst +++ b/docs/source/Eng/eng_index.rst @@ -51,6 +51,7 @@ Comprehensive guides for all AutoControl features. doc/new_features/v26_features_doc doc/new_features/v27_features_doc doc/new_features/v28_features_doc + doc/new_features/v29_features_doc doc/ocr_backends/ocr_backends_doc doc/observability/observability_doc doc/operations_layer/operations_layer_doc diff --git a/docs/source/Zh/doc/new_features/v29_features_doc.rst b/docs/source/Zh/doc/new_features/v29_features_doc.rst new file mode 100644 index 00000000..728c8c4a --- /dev/null +++ b/docs/source/Zh/doc/new_features/v29_features_doc.rst @@ -0,0 +1,28 @@ +========================================== +新功能 (2026-06-19) — 緩動拖曳 +========================================== + +沿曲線路徑的決定性緩動拖曳(PyAutoGUI 風格的 ``tween``),補足既有的 +人性化抖動。純標準庫;走完整五層。座標數學為純函式、可單元測試;派發 +透過可注入的 sink。 + +.. contents:: + :local: + :depth: 2 + + +用法 +==== + +:: + + from je_auto_control import tween_points, tween_drag, easing_names + + tween_points((0, 0), (100, 50), steps=20, easing="ease_out_cubic") + tween_drag((0, 0), (300, 200), steps=40, easing="ease_in_out_quad") + +``tween_points`` 回傳兩點之間 ``steps + 1`` 個緩動點;``tween_drag`` 在 +起點按下、沿各點移動、於終點放開。緩動函式:``linear`` / +``ease_in_out_quad`` / ``ease_out_cubic`` / ``ease_in_cubic``(見 +:func:`easing_names`)。對應 ``AC_tween_drag`` / ``ac_tween_drag`` +(``start`` / ``end`` 以 ``[x, y]`` 表示)。 diff --git a/docs/source/Zh/zh_index.rst b/docs/source/Zh/zh_index.rst index d063404a..c0b46b6b 100644 --- a/docs/source/Zh/zh_index.rst +++ b/docs/source/Zh/zh_index.rst @@ -51,6 +51,7 @@ AutoControl 所有功能的完整使用指南。 doc/new_features/v26_features_doc doc/new_features/v27_features_doc doc/new_features/v28_features_doc + doc/new_features/v29_features_doc doc/ocr_backends/ocr_backends_doc doc/observability/observability_doc doc/operations_layer/operations_layer_doc diff --git a/je_auto_control/__init__.py b/je_auto_control/__init__.py index 484155e7..2366d7b2 100644 --- a/je_auto_control/__init__.py +++ b/je_auto_control/__init__.py @@ -195,6 +195,10 @@ from je_auto_control.utils.process_doc import ( describe_step, generate_sop, write_sop, ) +# Eased / tweened interpolated drag +from je_auto_control.utils.tween_drag import ( + easing_names, tween_drag, tween_points, +) # Background popup/interrupt watchdog (unattended automation) from je_auto_control.utils.watchdog import ( PopupWatchdog, WatchdogRule, default_popup_watchdog, @@ -628,6 +632,7 @@ def start_autocontrol_gui(*args, **kwargs): "ClipboardHistory", "default_clipboard_history", "analyze_heal_log", "heal_stats", "scan_secrets", "describe_step", "generate_sop", "write_sop", + "easing_names", "tween_drag", "tween_points", # MCP server "AuditLogger", "HttpMCPServer", "MCPContent", "MCPPrompt", "MCPPromptArgument", "MCPResource", "MCPServer", "MCPTool", diff --git a/je_auto_control/gui/script_builder/command_schema.py b/je_auto_control/gui/script_builder/command_schema.py index 02bb56f4..be746322 100644 --- a/je_auto_control/gui/script_builder/command_schema.py +++ b/je_auto_control/gui/script_builder/command_schema.py @@ -667,6 +667,20 @@ def _add_misc_specs(specs: List[CommandSpec]) -> None: _add_resilience_specs(specs) _add_devex_specs(specs) _add_audit_specs(specs) + specs.append(CommandSpec( + "AC_tween_drag", "Mouse", "Tweened Drag", + fields=( + FieldSpec("steps", FieldType.INT, optional=True, default=30), + FieldSpec("easing", FieldType.ENUM, + choices=("linear", "ease_in_out_quad", "ease_out_cubic", + "ease_in_cubic"), + optional=True, default="ease_in_out_quad"), + FieldSpec("button", FieldType.ENUM, choices=_MOUSE_BUTTONS, + optional=True, default="mouse_left"), + ), + description="Drag along an eased path; 'start'/'end' [x,y] via JSON " + "view.", + )) specs.append(CommandSpec( "AC_generate_sop", "Report", "Generate SOP Document", fields=( diff --git a/je_auto_control/utils/executor/action_executor.py b/je_auto_control/utils/executor/action_executor.py index 0e27f033..38cd92d9 100644 --- a/je_auto_control/utils/executor/action_executor.py +++ b/je_auto_control/utils/executor/action_executor.py @@ -2872,6 +2872,16 @@ def _generate_sop(actions: List[Any], title: str = "Automation Procedure", return generate_sop(actions, title=title) +def _tween_drag(start: List[int], end: List[int], steps: int = 30, + easing: str = "ease_in_out_quad", + button: str = "mouse_left") -> Dict[str, Any]: + """Adapter: drag along an eased path from start to end.""" + from je_auto_control.utils.tween_drag import tween_drag + result = tween_drag(tuple(start), tuple(end), steps=int(steps), + easing=easing, button=button) + return {"points": result["points"]} + + class Executor: """ Executor @@ -3100,6 +3110,7 @@ def __init__(self): "AC_heal_stats": _heal_stats, "AC_scan_secrets": _scan_secrets, "AC_generate_sop": _generate_sop, + "AC_tween_drag": _tween_drag, "AC_a11y_record_start": _a11y_record_start, "AC_a11y_record_stop": _a11y_record_stop, "AC_a11y_record_events": _a11y_record_events, diff --git a/je_auto_control/utils/mcp_server/tools/_factories.py b/je_auto_control/utils/mcp_server/tools/_factories.py index 57afe694..8a92d078 100644 --- a/je_auto_control/utils/mcp_server/tools/_factories.py +++ b/je_auto_control/utils/mcp_server/tools/_factories.py @@ -2521,6 +2521,26 @@ def process_doc_tools() -> List[MCPTool]: ] +def tween_drag_tools() -> List[MCPTool]: + return [ + MCPTool( + name="ac_tween_drag", + description=("Drag from 'start' [x,y] to 'end' [x,y] along an " + "eased path (easing: linear / ease_in_out_quad / " + "ease_out_cubic / ease_in_cubic). Returns {points}."), + input_schema=schema({ + "start": {"type": "array", "items": {"type": "integer"}}, + "end": {"type": "array", "items": {"type": "integer"}}, + "steps": {"type": "integer"}, + "easing": {"type": "string"}, + "button": {"type": "string"}}, + required=["start", "end"]), + handler=h.tween_drag, + annotations=SIDE_EFFECT_ONLY, + ), + ] + + def unattended_tools() -> List[MCPTool]: return [ MCPTool( @@ -3578,7 +3598,7 @@ def media_assert_tools() -> List[MCPTool]: checkpoint_tools, set_of_marks_tools, screen_state_tools, input_macro_tools, resilience_tools, ci_annotation_tools, clipboard_history_tools, audit_analysis_tools, - process_doc_tools, + process_doc_tools, tween_drag_tools, screen_record_tools, process_and_shell_tools, remote_desktop_tools, gamepad_tools, usb_passthrough_tools, assertion_tools, data_source_tools, diff --git a/je_auto_control/utils/mcp_server/tools/_handlers.py b/je_auto_control/utils/mcp_server/tools/_handlers.py index 0a99de2c..69cb6fab 100644 --- a/je_auto_control/utils/mcp_server/tools/_handlers.py +++ b/je_auto_control/utils/mcp_server/tools/_handlers.py @@ -1229,6 +1229,13 @@ def generate_sop(actions, title="Automation Procedure", path=None): return _gen(actions, title=title) +def tween_drag(start, end, steps=30, easing="ease_in_out_quad", + button="mouse_left"): + from je_auto_control.utils.tween_drag import tween_drag as _td + return {"points": _td(tuple(start), tuple(end), steps=int(steps), + easing=easing, button=button)["points"]} + + def vlm_locate(description: str, screen_region: Optional[List[int]] = None, model: Optional[str] = None) -> Optional[List[int]]: diff --git a/je_auto_control/utils/tween_drag/__init__.py b/je_auto_control/utils/tween_drag/__init__.py new file mode 100644 index 00000000..af5fad54 --- /dev/null +++ b/je_auto_control/utils/tween_drag/__init__.py @@ -0,0 +1,6 @@ +"""Eased / tweened interpolated drag along a curved path.""" +from je_auto_control.utils.tween_drag.tween_drag import ( + easing_names, tween_drag, tween_points, +) + +__all__ = ["easing_names", "tween_drag", "tween_points"] diff --git a/je_auto_control/utils/tween_drag/tween_drag.py b/je_auto_control/utils/tween_drag/tween_drag.py new file mode 100644 index 00000000..a7abc6ff --- /dev/null +++ b/je_auto_control/utils/tween_drag/tween_drag.py @@ -0,0 +1,89 @@ +"""Eased / tweened interpolated drag along a curved path. + +AutoControl has humanized *jitter* but no deterministic named easings. +:func:`tween_points` produces an eased sequence of points between two +coordinates (pure math), and :func:`tween_drag` presses at the start, moves +through the points, and releases at the end — for smooth, deterministic +drags (PyAutoGUI-style ``tween``). + +The point math is pure and unit-testable; dispatch goes through an +injectable ``sink`` so the drag is tested without real input. Imports no +``PySide6``. +""" +from typing import Any, Callable, Dict, List, Optional, Tuple + + +def _linear(t: float) -> float: + return t + + +def _ease_in_out_quad(t: float) -> float: + return 2 * t * t if t < 0.5 else 1 - ((-2 * t + 2) ** 2) / 2 + + +def _ease_out_cubic(t: float) -> float: + return 1 - (1 - t) ** 3 + + +def _ease_in_cubic(t: float) -> float: + return t ** 3 + + +_EASINGS: Dict[str, Callable[[float], float]] = { + "linear": _linear, + "ease_in_out_quad": _ease_in_out_quad, + "ease_out_cubic": _ease_out_cubic, + "ease_in_cubic": _ease_in_cubic, +} + + +def easing_names() -> List[str]: + """Return the available easing-function names.""" + return sorted(_EASINGS) + + +def tween_points(start: Tuple[int, int], end: Tuple[int, int], + steps: int = 30, + easing: str = "ease_in_out_quad") -> List[List[int]]: + """Return ``steps + 1`` eased points from ``start`` to ``end``.""" + curve = _EASINGS.get(easing, _linear) + count = max(1, int(steps)) + start_x, start_y = start + end_x, end_y = end + points: List[List[int]] = [] + for index in range(count + 1): + progress = curve(index / count) + points.append([round(start_x + (end_x - start_x) * progress), + round(start_y + (end_y - start_y) * progress)]) + return points + + +def _default_sink(event: Dict[str, Any]) -> None: + from je_auto_control.wrapper.auto_control_mouse import ( + press_mouse, release_mouse, set_mouse_position) + x, y = int(event["x"]), int(event["y"]) + op = event["op"] + if op == "move": + set_mouse_position(x, y) + elif op == "press": + set_mouse_position(x, y) + press_mouse(event.get("button", "mouse_left"), x, y) + elif op == "release": + set_mouse_position(x, y) + release_mouse(event.get("button", "mouse_left"), x, y) + + +def tween_drag(start: Tuple[int, int], end: Tuple[int, int], *, + steps: int = 30, easing: str = "ease_in_out_quad", + button: str = "mouse_left", + sink: Optional[Callable[[Dict[str, Any]], None]] = None + ) -> Dict[str, Any]: + """Drag from ``start`` to ``end`` along an eased path; return point count.""" + points = tween_points(start, end, steps, easing) + dispatch = sink or _default_sink + first, last = points[0], points[-1] + dispatch({"op": "press", "button": button, "x": first[0], "y": first[1]}) + for x, y in points: + dispatch({"op": "move", "x": x, "y": y}) + dispatch({"op": "release", "button": button, "x": last[0], "y": last[1]}) + return {"points": len(points), "path": points} diff --git a/test/unit_test/headless/test_tween_drag_batch.py b/test/unit_test/headless/test_tween_drag_batch.py new file mode 100644 index 00000000..32401b2d --- /dev/null +++ b/test/unit_test/headless/test_tween_drag_batch.py @@ -0,0 +1,50 @@ +"""Headless tests for the eased / tweened drag. Pure math + injected sink +(no real mouse). Pure stdlib; no Qt imports.""" +import je_auto_control as ac +from je_auto_control.utils.tween_drag import ( + easing_names, tween_drag, tween_points) + + +def test_tween_points_endpoints_and_count(): + points = tween_points((0, 0), (100, 50), steps=10, easing="linear") + assert len(points) == 11 + assert points[0] == [0, 0] and points[-1] == [100, 50] + assert points[5] == [50, 25] # linear midpoint + + +def test_easing_changes_midpoint(): + linear = tween_points((0, 0), (100, 0), steps=10, easing="linear") + eased = tween_points((0, 0), (100, 0), steps=10, + easing="ease_in_out_quad") + # ease-in-out passes through the same midpoint but differs off-centre + assert eased[5] == [50, 0] + assert eased[2] != linear[2] + assert "ease_in_out_quad" in easing_names() + + +def test_tween_drag_dispatches_press_moves_release(): + events = [] + out = tween_drag((0, 0), (10, 10), steps=4, sink=events.append) + ops = [e["op"] for e in events] + assert ops[0] == "press" and ops[-1] == "release" + assert ops.count("move") == 5 # steps + 1 points + assert out["points"] == 5 + assert events[-1]["x"] == 10 and events[-1]["y"] == 10 + + +# --- wiring (registration only — executing moves the real mouse) --------- + +def test_wiring(): + known = ac.executor.known_commands() + assert "AC_tween_drag" in known + from je_auto_control.utils.mcp_server.tools import ( + build_default_tool_registry) + assert "ac_tween_drag" in {t.name for t in build_default_tool_registry()} + from je_auto_control.gui.script_builder.command_schema import _build_specs + assert "AC_tween_drag" in {s.command for s in _build_specs()} + + +def test_facade_exports(): + for attr in ("tween_points", "tween_drag", "easing_names"): + assert hasattr(ac, attr) + assert attr in ac.__all__