Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@

## Table of Contents

- [What's new (2026-06-19) — Heal Analytics & Secret Scan](#whats-new-2026-06-19--heal-analytics--secret-scan)
- [What's new (2026-06-19) — CI Annotations & Clipboard History](#whats-new-2026-06-19--ci-annotations--clipboard-history)
- [What's new (2026-06-19) — Resilience Primitives](#whats-new-2026-06-19--resilience-primitives)
- [What's new (2026-06-19) — Timed Input Macros](#whats-new-2026-06-19--timed-input-macros)
Expand Down Expand Up @@ -79,6 +80,13 @@

---

## What's new (2026-06-19) — Heal Analytics & Secret Scan

Two pure-stdlib audit/analysis tools. Full reference: [`docs/source/Eng/doc/new_features/v27_features_doc.rst`](docs/source/Eng/doc/new_features/v27_features_doc.rst).

- **Self-heal analytics** — `analyze_heal_log` / `heal_stats` (`AC_heal_stats`, `ac_heal_stats`): aggregate the self-heal log into heal-rate, strategy mix, fallback-rate, avg latency and the most-brittle locators — catch decaying selectors before they fail.
- **Secret scan** — `scan_secrets(data)` (`AC_scan_secrets`, `ac_scan_secrets`): flag hardcoded secrets in action JSON (by key name, value pattern, or high entropy) that should use `${secrets.*}`; vault refs ignored, previews masked.

## What's new (2026-06-19) — CI Annotations & Clipboard History

Two pure-stdlib utilities. Full reference: [`docs/source/Eng/doc/new_features/v26_features_doc.rst`](docs/source/Eng/doc/new_features/v26_features_doc.rst).
Expand Down
8 changes: 8 additions & 0 deletions README/README_zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@

## 目录

- [本次更新 (2026-06-19) — 修复分析与机密扫描](#本次更新-2026-06-19--修复分析与机密扫描)
- [本次更新 (2026-06-19) — CI 注解与剪贴板历史](#本次更新-2026-06-19--ci-注解与剪贴板历史)
- [本次更新 (2026-06-19) — 韧性原语](#本次更新-2026-06-19--韧性原语)
- [本次更新 (2026-06-19) — 计时输入宏](#本次更新-2026-06-19--计时输入宏)
Expand Down Expand Up @@ -78,6 +79,13 @@

---

## 本次更新 (2026-06-19) — 修复分析与机密扫描

两项纯标准库的审计/分析工具。完整参考:[`docs/source/Zh/doc/new_features/v27_features_doc.rst`](../docs/source/Zh/doc/new_features/v27_features_doc.rst)。

- **自我修复分析** — `analyze_heal_log` / `heal_stats`(`AC_heal_stats`、`ac_heal_stats`):把自我修复记录汇总成 heal-rate、策略组合、fallback-rate、平均延迟与最脆弱定位器——在选择器衰退失效前抓出来。
- **机密扫描** — `scan_secrets(data)`(`AC_scan_secrets`、`ac_scan_secrets`):标记 action JSON 中应改用 `${secrets.*}` 的硬编码机密(依键名、值样式或高熵);保险库引用会略过、预览掩码。

## 本次更新 (2026-06-19) — CI 注解与剪贴板历史

两项纯标准库工具。完整参考:[`docs/source/Zh/doc/new_features/v26_features_doc.rst`](../docs/source/Zh/doc/new_features/v26_features_doc.rst)。
Expand Down
8 changes: 8 additions & 0 deletions README/README_zh-TW.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@

## 目錄

- [本次更新 (2026-06-19) — 修復分析與機密掃描](#本次更新-2026-06-19--修復分析與機密掃描)
- [本次更新 (2026-06-19) — CI 註解與剪貼簿歷史](#本次更新-2026-06-19--ci-註解與剪貼簿歷史)
- [本次更新 (2026-06-19) — 韌性原語](#本次更新-2026-06-19--韌性原語)
- [本次更新 (2026-06-19) — 計時輸入巨集](#本次更新-2026-06-19--計時輸入巨集)
Expand Down Expand Up @@ -78,6 +79,13 @@

---

## 本次更新 (2026-06-19) — 修復分析與機密掃描

兩項純標準庫的稽核/分析工具。完整參考:[`docs/source/Zh/doc/new_features/v27_features_doc.rst`](../docs/source/Zh/doc/new_features/v27_features_doc.rst)。

- **自我修復分析** — `analyze_heal_log` / `heal_stats`(`AC_heal_stats`、`ac_heal_stats`):把自我修復記錄彙總成 heal-rate、策略組合、fallback-rate、平均延遲與最脆弱定位器——在選擇器衰退失效前抓出來。
- **機密掃描** — `scan_secrets(data)`(`AC_scan_secrets`、`ac_scan_secrets`):標記 action JSON 中應改用 `${secrets.*}` 的寫死機密(依鍵名、值樣式或高熵);保險庫引用會略過、預覽遮罩。

## 本次更新 (2026-06-19) — CI 註解與剪貼簿歷史

兩項純標準庫工具。完整參考:[`docs/source/Zh/doc/new_features/v26_features_doc.rst`](../docs/source/Zh/doc/new_features/v26_features_doc.rst)。
Expand Down
44 changes: 44 additions & 0 deletions docs/source/Eng/doc/new_features/v27_features_doc.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
==================================================
New Features (2026-06-19) — Heal Analytics & Secret Scan
==================================================

Two pure-standard-library audit/analysis tools: aggregate the self-healing
log into drift metrics, and scan action JSON for hardcoded secrets. Full
stack.

.. contents::
:local:
:depth: 2


Self-heal analytics
==================

::

from je_auto_control import analyze_heal_log, heal_stats

analyze_heal_log(limit=200) # over the live self-heal log
heal_stats(events) # over a supplied event list

Aggregates self-heal events into ``{total, healed, heal_rate, by_method,
fallbacks, fallback_rate, avg_duration_ms, top_brittle}`` — surfacing
locators that increasingly need the VLM fallback (decaying selectors) before
they fail. Exposed as ``AC_heal_stats`` / ``ac_heal_stats``.


Secret scan
==========

::

from je_auto_control import scan_secrets

scan_secrets(action_json) # [{path, kind, preview}, ...]

Walks a JSON-like structure and flags string values that look like secrets —
by key name (``password`` / ``token`` / ``api_key`` …), by value pattern
(AWS / GitHub tokens, private-key blocks), or by high Shannon entropy — that
should reference the vault (``${secrets.NAME}``). Values already referencing
the vault are ignored; previews are masked. Exposed as ``AC_scan_secrets`` /
``ac_scan_secrets``.
1 change: 1 addition & 0 deletions docs/source/Eng/eng_index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ Comprehensive guides for all AutoControl features.
doc/new_features/v24_features_doc
doc/new_features/v25_features_doc
doc/new_features/v26_features_doc
doc/new_features/v27_features_doc
doc/ocr_backends/ocr_backends_doc
doc/observability/observability_doc
doc/operations_layer/operations_layer_doc
Expand Down
41 changes: 41 additions & 0 deletions docs/source/Zh/doc/new_features/v27_features_doc.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
==================================================
新功能 (2026-06-19) — 修復分析與機密掃描
==================================================

兩項純標準庫的稽核/分析工具:把自我修復記錄彙總成漂移指標,以及掃描
action JSON 中的寫死機密。走完整五層。

.. contents::
:local:
:depth: 2


自我修復分析
============

::

from je_auto_control import analyze_heal_log, heal_stats

analyze_heal_log(limit=200) # 針對即時自我修復記錄
heal_stats(events) # 針對提供的事件清單

把自我修復事件彙總成 ``{total, healed, heal_rate, by_method, fallbacks,
fallback_rate, avg_duration_ms, top_brittle}``——在定位器真正失效前,揪出
越來越需要 VLM 後備(衰退中的選擇器)的那些。對應 ``AC_heal_stats`` /
``ac_heal_stats``。


機密掃描
========

::

from je_auto_control import scan_secrets

scan_secrets(action_json) # [{path, kind, preview}, ...]

走訪 JSON 結構並標記看起來像機密的字串值——依鍵名(``password`` /
``token`` / ``api_key`` …)、依值樣式(AWS / GitHub token、私鑰區塊),或
依高夏農熵——這些應改用保險庫(``${secrets.NAME}``)。已引用保險庫的值會被
略過;預覽會遮罩。對應 ``AC_scan_secrets`` / ``ac_scan_secrets``。
1 change: 1 addition & 0 deletions docs/source/Zh/zh_index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ AutoControl 所有功能的完整使用指南。
doc/new_features/v24_features_doc
doc/new_features/v25_features_doc
doc/new_features/v26_features_doc
doc/new_features/v27_features_doc
doc/ocr_backends/ocr_backends_doc
doc/observability/observability_doc
doc/operations_layer/operations_layer_doc
Expand Down
4 changes: 4 additions & 0 deletions je_auto_control/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -188,6 +188,9 @@
from je_auto_control.utils.clipboard_history import (
ClipboardHistory, default_clipboard_history,
)
# Self-heal analytics + action-secrets scanning (audit/analysis)
from je_auto_control.utils.heal_analytics import analyze_heal_log, heal_stats
from je_auto_control.utils.secrets_scan import scan_secrets
# Background popup/interrupt watchdog (unattended automation)
from je_auto_control.utils.watchdog import (
PopupWatchdog, WatchdogRule, default_popup_watchdog,
Expand Down Expand Up @@ -619,6 +622,7 @@ def start_autocontrol_gui(*args, **kwargs):
"CircuitBreaker", "CircuitOpenError", "RetryPolicy", "retry_call",
"emit_annotations", "format_annotation",
"ClipboardHistory", "default_clipboard_history",
"analyze_heal_log", "heal_stats", "scan_secrets",
# MCP server
"AuditLogger", "HttpMCPServer", "MCPContent", "MCPPrompt",
"MCPPromptArgument", "MCPResource", "MCPServer", "MCPTool",
Expand Down
16 changes: 16 additions & 0 deletions je_auto_control/gui/script_builder/command_schema.py
Original file line number Diff line number Diff line change
Expand Up @@ -666,6 +666,22 @@ def _add_misc_specs(specs: List[CommandSpec]) -> None:
_add_input_macro_specs(specs)
_add_resilience_specs(specs)
_add_devex_specs(specs)
_add_audit_specs(specs)


def _add_audit_specs(specs: List[CommandSpec]) -> None:
specs.append(CommandSpec(
"AC_heal_stats", "Testing", "Self-Heal Analytics",
fields=(FieldSpec("limit", FieldType.INT, optional=True,
default=200),),
description="Aggregate the self-heal log (heal rate, brittle "
"locators).",
))
specs.append(CommandSpec(
"AC_scan_secrets", "Tools", "Scan for Hardcoded Secrets",
description="Scan 'data' (JSON view) for hardcoded secrets that "
"should use ${secrets.*}.",
))


def _add_devex_specs(specs: List[CommandSpec]) -> None:
Expand Down
14 changes: 14 additions & 0 deletions je_auto_control/utils/executor/action_executor.py
Original file line number Diff line number Diff line change
Expand Up @@ -2851,6 +2851,18 @@ def _clip_history_stop() -> Dict[str, Any]:
return {"running": default_clipboard_history.running}


def _heal_stats(limit: int = 200) -> Dict[str, Any]:
"""Adapter: aggregate the self-heal log into metrics."""
from je_auto_control.utils.heal_analytics import analyze_heal_log
return analyze_heal_log(limit=int(limit))


def _scan_secrets(data: Any) -> Dict[str, Any]:
"""Adapter: scan JSON/data for hardcoded secrets."""
from je_auto_control.utils.secrets_scan import scan_secrets
return {"findings": scan_secrets(data)}


class Executor:
"""
Executor
Expand Down Expand Up @@ -3076,6 +3088,8 @@ def __init__(self):
"AC_clip_history_search": _clip_history_search,
"AC_clip_history_start": _clip_history_start,
"AC_clip_history_stop": _clip_history_stop,
"AC_heal_stats": _heal_stats,
"AC_scan_secrets": _scan_secrets,
"AC_a11y_record_start": _a11y_record_start,
"AC_a11y_record_stop": _a11y_record_stop,
"AC_a11y_record_events": _a11y_record_events,
Expand Down
6 changes: 6 additions & 0 deletions je_auto_control/utils/heal_analytics/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
"""Analytics over the self-healing event log (heal rate, brittle locators)."""
from je_auto_control.utils.heal_analytics.heal_analytics import (
analyze_heal_log, heal_stats,
)

__all__ = ["analyze_heal_log", "heal_stats"]
71 changes: 71 additions & 0 deletions je_auto_control/utils/heal_analytics/heal_analytics.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
"""Analytics over the self-healing event log.

Self-healing without analytics hides accumulating UI drift. This aggregates
the :class:`HealEventLog` into metrics — heal rate, strategy mix, fallback
rate (image template failed, fell back to the VLM), average latency, and the
most-brittle locators — so decaying selectors are surfaced before they fail
outright.

Pure standard library; the log is imported lazily so :func:`heal_stats`
(over supplied events) is unit-testable without any log file.
"""
from typing import Any, Dict, List, Tuple


def _attr(event: Any, name: str) -> Any:
if isinstance(event, dict):
return event.get(name)
return getattr(event, name, None)


def _accumulate(events: List[Any]) -> Tuple[Dict[str, int], Dict[str, int],
List[float], int]:
by_method: Dict[str, int] = {}
brittle: Dict[str, int] = {}
durations: List[float] = []
fallbacks = 0
for event in events:
method = _attr(event, "method") or "?"
by_method[method] = by_method.get(method, 0) + 1
if _attr(event, "image_error"):
fallbacks += 1
key = (_attr(event, "template_path")
or _attr(event, "description") or "?")
brittle[key] = brittle.get(key, 0) + 1
duration = _attr(event, "duration_ms")
if duration is not None:
durations.append(float(duration))
return by_method, brittle, durations, fallbacks


def heal_stats(events: List[Any]) -> Dict[str, Any]:
"""Aggregate self-heal events into a metrics dict.

Each event is a :class:`HealEvent` or dict with ``method`` /
``coordinates`` / ``duration_ms`` / ``image_error`` /
``template_path`` / ``description``.
"""
events = list(events)
total = len(events)
healed = sum(1 for e in events if _attr(e, "coordinates") is not None)
by_method, brittle, durations, fallbacks = _accumulate(events)
top_brittle = sorted(brittle.items(), key=lambda kv: (-kv[1], kv[0]))[:5]
return {
"total": total, "healed": healed,
"heal_rate": round(healed / total, 4) if total else 0.0,
"by_method": by_method,
"fallbacks": fallbacks,
"fallback_rate": round(fallbacks / total, 4) if total else 0.0,
"avg_duration_ms": (round(sum(durations) / len(durations), 2)
if durations else 0.0),
"top_brittle": [{"locator": key, "fallbacks": count}
for key, count in top_brittle],
}


def analyze_heal_log(limit: int = 200, log: Any = None) -> Dict[str, Any]:
"""Aggregate the most recent events from the self-heal log."""
if log is None:
from je_auto_control.utils.self_healing import default_heal_log
log = default_heal_log
return heal_stats(log.list_events(limit=int(limit)))
26 changes: 25 additions & 1 deletion je_auto_control/utils/mcp_server/tools/_factories.py
Original file line number Diff line number Diff line change
Expand Up @@ -2478,6 +2478,30 @@ def clipboard_history_tools() -> List[MCPTool]:
]


def audit_analysis_tools() -> List[MCPTool]:
return [
MCPTool(
name="ac_heal_stats",
description=("Aggregate the self-healing event log into metrics: "
"heal_rate, by_method, fallback_rate, avg latency, "
"and the most-brittle locators."),
input_schema=schema({"limit": {"type": "integer"}}),
handler=h.heal_stats,
annotations=READ_ONLY,
),
MCPTool(
name="ac_scan_secrets",
description=("Scan a JSON/data structure for hardcoded secrets "
"(by key name, value pattern — AWS/GitHub/private-key "
"— or high entropy) that should use ${secrets.*}. "
"Returns masked {findings}."),
input_schema=schema({"data": {}}, required=["data"]),
handler=h.scan_secrets,
annotations=READ_ONLY,
),
]


def unattended_tools() -> List[MCPTool]:
return [
MCPTool(
Expand Down Expand Up @@ -3534,7 +3558,7 @@ def media_assert_tools() -> List[MCPTool]:
sbom_tools, sharding_tools, data_quality_tools, i18n_tools,
checkpoint_tools, set_of_marks_tools, screen_state_tools,
input_macro_tools, resilience_tools,
ci_annotation_tools, clipboard_history_tools,
ci_annotation_tools, clipboard_history_tools, audit_analysis_tools,
screen_record_tools,
process_and_shell_tools, remote_desktop_tools, gamepad_tools,
usb_passthrough_tools, assertion_tools, data_source_tools,
Expand Down
10 changes: 10 additions & 0 deletions je_auto_control/utils/mcp_server/tools/_handlers.py
Original file line number Diff line number Diff line change
Expand Up @@ -1211,6 +1211,16 @@ def clip_history_stop():
return {"running": default_clipboard_history.running}


def heal_stats(limit=200):
from je_auto_control.utils.heal_analytics import analyze_heal_log
return analyze_heal_log(limit=int(limit))


def scan_secrets(data):
from je_auto_control.utils.secrets_scan import scan_secrets as _scan
return {"findings": _scan(data)}


def vlm_locate(description: str,
screen_region: Optional[List[int]] = None,
model: Optional[str] = None) -> Optional[List[int]]:
Expand Down
4 changes: 4 additions & 0 deletions je_auto_control/utils/secrets_scan/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
"""Scan action JSON / data for hardcoded secrets that should be vaulted."""
from je_auto_control.utils.secrets_scan.secrets_scan import scan_secrets

__all__ = ["scan_secrets"]
Loading
Loading