feat(skills): add test-hardening skill and plugin manifest

[h30s]

Summary

Closes #4

Adds the new test-hardening skill, manifest, and marketplace entry.

Most test advice just tells you to increase coverage numbers. This skill forces a simple discipline: break the code on purpose, watch the exact test fail by name, and restore the code. If the test doesn't fail when the code is broken, it's not actually protecting anything.

What task was this used on, and what changed in the answer?

• Task: Checking test suites that had 100% green runs but weak assertions—like guards hidden behind unreachable branches, fixtures masking real checks, and assertions that just mirrored whatever the code returned.
• What changed: Instead of giving generic advice about adding coverage, the model now formulates a concrete mutation first ("I will change X to Y, and I expect <test name> to fail"), checks if that exact test failed by name, and flags any surviving mutation as an issue.

What was deliberately left out, and why?

1. Auto-fixing or generating large test files: The skill is set to tools: chat (read-only review mode). Its job is to find weak tests and guide you, not generate mounds of unverified test code.
2. Treating mutation as a magic bullet: Passing mutation tests only proves a test can fail when code changes—it doesn't prove the test is asserting the right business logic (a test can still pin a bug in place). We explicitly called out this limit so the model doesn't give false confidence.

[ms-shashank]

This one line stops the skill being able to do what it describes.

In src/skills/format.ts the toolset vocabulary is 'full' | 'chat' | 'plan', and chat is the read-only one. The validator's own warning text spells it out: an unrecognised value falls back to chat and the message reads "so this skill runs read-only".

The core discipline here is Mutate, Observe, Restore, and all three need write access or command execution:

• Mutate: "Apply a targeted mutation" needs a file edit
• Observe: "Run or evaluate the tests and identify the failure BY NAME" needs to run the suite
• Restore: "Immediately restore the code to its clean state" needs a file edit

Under chat the model can do none of those, so it has to imagine all three. Then the Output section asks it to fill in "Hardened Tests: the list of tests verified with deliberate mutations, naming the exact mutation applied and the named test failure observed". A read-only model filling that in is reporting verification it did not perform, which is the exact failure this skill was written to catch.

Suggested change

	tools: chat
	tools: full

The reason you gave in the PR description is a good one and I do not want to lose it. Not generating mounds of unverified test code is right. That belongs in the body as an instruction rather than in the toolset, because the toolset takes away the ability to verify at the same time as the ability to write. Add a line saying the skill does not author new test files, it mutates existing code temporarily and restores it.

Everything else stands. "Passing is the null result" is the right opening, and the worked example that ends "The test failed on mutation, but the test is still wrong" is the best paragraph in the file, because it stops the skill overselling itself.

[ms-shashank]

I have reviewed this. The writing is genuinely strong and I want it in, but there is one line in the frontmatter that makes the skill unable to do the thing it describes.

The frontmatter says tools: chat. In src/skills/format.ts the toolset vocabulary is 'full' | 'chat' | 'plan', and chat is the read-only one. The validator's own warning text spells it out: an unrecognised value falls back to chat and the message reads "so this skill runs read-only".

The core discipline in this skill is Mutate, Observe, Restore. All three need write access or command execution:

Mutate: "Apply a targeted mutation" needs a file edit
Observe: "Run or evaluate the tests and identify the failure BY NAME" needs to run the suite
Restore: "Immediately restore the code to its clean state" needs a file edit
Under chat the model can do none of those, so it would have to imagine all three. Then the required output section asks it to fill in "Hardened Tests: the list of tests verified with deliberate mutations, naming the exact mutation applied and the named test failure observed". A read-only model filling in that section is reporting verification it did not perform, which is the exact failure this skill was written to catch.

The reason you gave for chat in the PR description is a good one and I do not want to lose it. Not generating mounds of unverified test code is right. That belongs in the body as an instruction rather than in the toolset, because the toolset removes the ability to verify at the same time as it removes the ability to write.

So: tools: full, and add a line to the body saying the skill does not author new test files, it mutates existing code temporarily and restores it.

Everything else stands. "Passing is the null result" is the right opening. The worked example that ends "The test failed on mutation, but the test is still wrong" is the best paragraph in the file, because it stops the skill overselling itself.

Rebase on top of #12 when you push, the marketplace entry will conflict.

[h30s]

@ms-shashank done!

[ms-shashank]

tools: full is in and that was the blocker, so the content is good to go.

Holding the merge on the red check rather than overriding it. The check job is failing and I could not read the step log from the run, so I am not going to guess at which assertion tripped.

Most likely cause: this branch was cut before #12 merged, so main has moved. plugins/ on main now holds code-review, triage and ui-review, and marketplace.json lists all three. Your branch adds a fourth listing against the older file. Rebase on main, confirm marketplace.json lists all four and that each source points at a directory that exists, and push.

If it is still red after the rebase, paste the failing step here and I will look at it with you.

Two of the checks in that workflow are worth knowing about generally, because they are easy to trip and the reason is not obvious:

• No oxcode:final-override blocks. An installed skill never receives that slot, so a skill written with one silently does nothing at runtime. The check exists so you find that out from CI rather than from a skill that appears to work and does not.
• No model or provider named. Model ids change under us, so a skill that names one goes stale for everybody at once. capability is the field to use and OxCode does the routing.

Both worth a paragraph each in SKILL_STYLE.md when you get to #3.

[h30s]

@ms-shashank I checked why the check failed. The tests didn't actually fail.

GitHub Actions couldn't start the test runner because of an account billing issue:
The job was not started because your account is locked due to a billing issue.
(The same thing happened on main right after #12 was merged).

I checked everything locally on my machine and all 5 tests passed:

• The branch is already updated with main (after feat(skills): add triage skill and plugin manifest #12).
• marketplace.json has all 4 plugins listed correctly.
• All folders and SKILL.md files are in place with the right settings (tools: full, no models named, and no override blocks).

Everything is ready to merge whenever you're ready!

[ms-shashank]

Correcting myself: I told you to rebase and paste the failing step, and the red has nothing to do with your code. It is a billing issue on the org, so the runner never starts, which is why the job reports failure in a couple of seconds with no steps at all. Sorry for sending you looking for something that was not there, and for the three PRs that sat behind it.

tools: full was the real blocker on this one and that is fixed. Merging.