Skip to content

[0.1.7] Add GUI QA driver - #62

Merged
unurgunite merged 4 commits into
v0.1.7from
feature/rubymine-qa-tests
Sep 18, 2026
Merged

unurgunite merged 4 commits into
v0.1.7from
feature/rubymine-qa-tests

Conversation

@unurgunite

@unurgunite unurgunite commented Sep 17, 2026 •

Copy link
Copy Markdown
Member

Description

Adds a host-side GUI test driver (tools/gui-qa/) that drives the plugin inside RubyMine running in a Tart VM and verifies behavior end to end: palette/menu actions, notification balloons, editor content, and idea.log oracles. Covers sections 3D (RBS detection), 3E (Update Types), 3F (workspace batch), 3G (Doctor), and 3H (MCP/settings/folding/i18n) — driver and docs only, no production code.

How it works:

  • lib.sh — SSH transport (VM IP resolved per run, proxy vars stripped), screenshots over scp, Vision OCR assertions, Search Everywhere handling (opened via menu clicks — keyboard chords get intercepted by the macOS man-page service and steal focus), frontmost-app gates before every keystroke batch.
  • stand.sh / ws.sh / menu.sh / anno.sh / ut.sh / settings.sh — project-stand bootstrap, workspace-run helper (log-gated: waits for a fresh collectRubyFiles line before accepting a balloon, so stale notifications can't false-pass), context-menu firing, annotator log oracles, Update Types helpers, Settings helpers (triple-click label + Space for checkboxes — the square has no OCR row; clipboard reads via in-balloon drag, not Cmd+A).
  • cases/3*.sh — one script per checklist item, each ending in ok / not ok with md5/log/balloon oracles.
  • merge.py — merges case outcomes into qa-results-auto.json by section order.

Results (recorded in qa-results-auto.json in the gem repo, 192 ok / 7 fail / 3 skip of 202): 3D 7/7, 3E 8/8, 3F 5/6, 3G 5/5, 3H 6/8. Remaining items are documented scope cuts or drifts in the case files: no live MCP port in the VM (3H.2, registration + code shape asserted instead), 33- and 207-file workspace checks finish too fast for any live cancel (3F.4, cancel plumbing stays unit-tested), warn-toggle OFF still flags syntax-broken YARD (3H.5 — gem always emits invalid_type, toggle not wired into CHECK), plus the pre-existing 3C gem drifts.

Also includes RUBYMINE-PILOT.md with the pilot findings the driver is built on.

Related Issue

N/A — test infrastructure, no user-facing change.

Checklist

  • Driver executed live against RubyMine 2026.2 (3D 7/7, 3E 8/8, 3F 5/6, 3G 5/5, 3H 6/8; outcomes merged into qa-results-auto.json)
  • Tests pass (./gradlew test) — N/A, no production code touched
  • Formatting clean (./gradlew spotlessCheck) — N/A, shell/markdown only
  • Static analysis clean (./gradlew detekt) — N/A
  • Plugin verified (./gradlew verifyPlugin) — N/A
  • CHANGELOG updated — N/A, no user-facing change

Screenshots (if applicable)

N/A — assertions are OCR text plus logs; shots stay in /tmp/gui-qa-rm/ by design and are never committed.

@unurgunite unurgunite self-assigned this Sep 17, 2026
@unurgunite unurgunite added the enhancement New feature or request label Sep 17, 2026
@unurgunite
unurgunite merged commit 99bcd46 into v0.1.7 Sep 18, 2026
1 check passed
@unurgunite
unurgunite deleted the feature/rubymine-qa-tests branch September 18, 2026 07:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant