PLTF-2046: add test-improvement-workflow skills

[aivong-openhands]

Summary

• Adds skills/test-improvement-workflow/SKILL.md as a concise top-level orchestrator for improving test suites with Dave Farley's 8 properties
• Splits the detailed guidance into three focused reference skills:
  • skills/test-prioritization-framework/SKILL.md
  • skills/test-validation-checklist/SKILL.md
  • skills/test-improvement-patterns/SKILL.md
• Registers all four skills in marketplaces/openhands-extensions.json under the testing category
• Runs sync_extensions.py to update the README catalog (51 total extensions, 42 skills)

Skill Overview

The test-improvement-workflow skill now stays small and progressive-disclosure-friendly by delegating deeper guidance to three focused skills:

1. Audit - Run test-design-reviewer to get a Farley Score
2. Prioritize - Open test-prioritization-framework to classify issues into CRITICAL / HIGH / MEDIUM tiers
3. Present - Always show the prioritized improvements table, even for exemplary suites
4. Validate - Open test-validation-checklist to verify each improvement addresses a real code issue
5. Plan - Assign the right companion skill to each approved task
6. Execute - Open test-improvement-patterns for safe execution loops and common fix patterns
7. Verify - Re-audit in a new conversation for unbiased evaluation

Triggers on: test improvement workflow, improve test quality, refactor tests, audit tests and fix them

Test plan

• uv sync --group test
• uv run python scripts/sync_extensions.py --check
• uv run pytest -q tests/test_sync_extensions.py tests/test_skill_plugin_loading.py tests/test_skills_have_readme.py tests/test_workflow_sync.py
• Each new skill has README.md, .plugin/plugin.json, and vendor symlinks
• The main orchestrator skill is reduced from 603 lines to 103 lines

This PR description was updated by an AI agent (OpenHands) on behalf of aivong-openhands.

[aivong-openhands]

@OpenHands follow up on failing PR checks

[openhands-ai]

I'm on it! aivong-openhands can track my progress at all-hands.dev

[aivong-openhands]

Followed up on the failing checks and pushed fix commit 8ac0f2d to add-test-improvement-workflow.

What I fixed:

• added the missing skills/test-improvement-workflow/README.md
• added skills/test-improvement-workflow/.plugin/plugin.json
• ran scripts/sync_extensions.py to refresh the README catalog and generate the expected plugin symlinks

Verification:

• uv run --group test python scripts/sync_extensions.py --check
• uv run --group test pytest tests/ -q
• GitHub PR checks are now all passing

This comment was created by an AI agent (OpenHands) on behalf of the user.

[openhands-ai]

Final summary of new work since my last update:

• Completed the PR follow-up fully.
• Confirmed the fix commit 8ac0f2d was pushed to add-test-improvement-workflow.
• Waited for GitHub Actions and verified all PR checks passed.
• Corrected the malformed PR comment and updated it with a clean summary of the fixes.
• Confirmed the working tree is clean afterward.

Conciseness check:

• No additional code changes were made beyond what was needed.
• The only post-fix action was repairing the PR comment text; no extraneous repository changes were introduced.

[all-hands-bot]

🔴 Needs improvement - Violates fundamental principles

This skill has solid content and addresses a real need (test quality improvement), but violates two core repository principles: it depends on skills that don't exist in this repo, and it violates the progressive disclosure principle at 590 lines.

See inline comments for details.

[all-hands-bot]

Review Summary

[CRITICAL ISSUES] (Must fix - these break fundamental principles)

• [skills/test-improvement-workflow/SKILL.md, Line 28] Broken Dependencies: References four skills from an external repository (Paul Hammond's dotfiles) that don't exist in OpenHands/extensions. The skill cannot function without these prerequisites.

• [skills/test-improvement-workflow/SKILL.md, Lines 18-23] Invalid Dependencies: The frontmatter declares dependencies on test-design-reviewer, tdd, refactoring, and testing - none of which exist in this repository.

[IMPROVEMENT OPPORTUNITIES] (Should fix - violates good taste)

• [skills/test-improvement-workflow/SKILL.md, Overall] Progressive Disclosure Violation: At 590 lines, this violates the AgentSkills principle of progressive disclosure. The skill should be split into:
  • Main orchestrator (~50-100 lines)
  • Separate reference skills for prioritization framework, validation checklist, and improvement patterns

[RISK ASSESSMENT]

⚠️ Risk Assessment: 🟡 MEDIUM

This PR adds a new skill without modifying existing functionality, so blast radius is limited. However, it introduces broken dependencies that will prevent the skill from working, and violates architectural principles around skill design (progressive disclosure, composability).

Risk factors:

• Pattern conformance: Violates the progressive disclosure pattern documented in AGENTS.md
• Dependency integrity: Creates broken references to external skills not in this repository
• Blast radius: Low - isolated to new skill addition, doesn't affect existing extensions

Recommendation: Address the critical dependency issues before merging. Either import the dependent skills or make this skill self-contained/properly documented as requiring external dependencies.

---

VERDICT:

❌ Needs rework: Fundamental design issues must be addressed first

KEY INSIGHT:

This skill attempts to orchestrate external dependencies that don't exist in the repository, creating a broken user experience where the skill appears available but cannot function.

---

Improve this review? If any feedback above seems incorrect or irrelevant to this repository, you can teach the reviewer to do better:

1. Add a .agents/skills/custom-codereview-guide.md file to your branch (or edit it if one already exists) with the /codereview trigger and the context the reviewer is missing (e.g., "Security concerns about X do not apply here because Y"). See the customization docs for the required frontmatter format.
2. Re-request a review - the reviewer reads guidelines from the PR branch, so your changes take effect immediately.
3. When your PR is merged, the guideline file goes through normal code review by repository maintainers.

Resolve with AI? Install the iterate skill in your agent and run /iterate to automatically drive this PR through CI, review, and QA until it's merge-ready.

[all-hands-bot]

🟢 Good taste - Elegant solution to the progressive disclosure problem.

Successfully transformed an oversized 603-line skill into a clean orchestrator pattern (103 lines) that delegates to three focused reference skills. External dependencies (Paul Hammond's skills) are properly documented with source URLs rather than bundled.

[RISK ASSESSMENT]

• [Overall PR] ⚠️ Risk Assessment: 🟢 LOW
  • Documentation/content change adding new skills
  • No breaking changes to existing functionality
  • External dependencies clearly documented as prerequisites
  • All required files present (README.md, .plugin/plugin.json, symlinks)
  • Test plan shows successful validation

VERDICT:
✅ Worth merging - Addresses previous critical feedback and follows all repository conventions

KEY INSIGHT:
This is exactly what progressive disclosure should look like - a concise entry point that delegates to focused reference materials when deeper guidance is needed.

[xingyaoww]

@aivong-openhands I wonder if we can just create ONE plugin that bundles all these skills in it? 👀

[aivong-openhands]

@aivong-openhands I wonder if we can just create ONE plugin that bundles all these skills in it? 👀

What is the difference between a plugin and a skill in this repo?