Skip to content

Add coordinate-space mapping (model grid <-> physical pixels)#253

Merged
JE-Chen merged 1 commit into
devfrom
feat/coordinate-space-batch
Jun 20, 2026
Merged

Add coordinate-space mapping (model grid <-> physical pixels)#253
JE-Chen merged 1 commit into
devfrom
feat/coordinate-space-batch

Conversation

@JE-Chen

@JE-Chen JE-Chen commented Jun 20, 2026

Copy link
Copy Markdown
Member

Computer-use agent batch (from multi-agent web research; verified absent). Full layers + tests + EN/Zh v45 docs + README.

Feature (utils/coordinate_space, pure-arithmetic + Pillow)

  • CoordinateSpace maps a model grid <-> physical pixels both ways (round + clamp). normalized_space (Gemini-style square 1000x1000), xga_space (Anthropic-style downscale, aspect-preserved, never upscales), downscale_png (resize a screenshot to the model's input size via Pillow — already core).
  • Why: every 2026 computer-use/VLA model clicks in a fixed virtual grid, not physical pixels — the repo had no observe/act transform for this (only a replay-time scale_coordinates multiplier). Closes that gap so agent backends can right-size screenshots and translate clicks back.
  • Executor AC_to_physical / AC_to_model; MCP ac_*; Builder under Agent.

Verification

  • 9 tests pass (normalized round-trip, grid scaling + clamp, XGA aspect/no-upscale, clamping bounds, PNG downscale under importorskip, executor round-trip, wiring, facade); ruff clean; radon no CC≥C; bandit clean; PySide6-free.

@codacy-production

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 24 complexity · 0 duplication

Metric Results
Complexity 24
Duplication 0

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@JE-Chen JE-Chen merged commit 952bb4a into dev Jun 20, 2026
16 checks passed
@JE-Chen JE-Chen deleted the feat/coordinate-space-batch branch June 20, 2026 05:44
@sonarqubecloud

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant