Skip to content

feat(mcp): 6 spec-driven authoring tools (resolve_step, get_scenario/step_definition, step_catalog, list_tags, project_dependencies) - #1

Merged
Karzone merged 7 commits into
mainfrom
feat/mcp-resolve-step-unbound-steps
Jul 29, 2026
Merged

feat(mcp): 6 spec-driven authoring tools (resolve_step, get_scenario/step_definition, step_catalog, list_tags, project_dependencies)#1
Karzone merged 7 commits into
mainfrom
feat/mcp-resolve-step-unbound-steps

Conversation

@Karzone

@Karzone Karzone commented Jul 29, 2026

Copy link
Copy Markdown
Owner

Adds 6 read-only MCP tools that ground an AI agent in what already exists when authoring/maintaining BDD tests — reuse-first, not regenerate. Takes the server from 5 → 11 tools.

Tools

  • resolve_step — resolve a Gherkin phrase to the existing definition(s) that would bind it, via the real StepMatcher (regex/cucumber, keyword-agnostic). exact / ambiguous / none; on none, returns existing steps ranked by shared terms (a near-miss that swaps a word still surfaces the closest steps).
  • get_scenario — full scenario detail (feature, tags, kind, ordered steps) by name.
  • get_step_definition — step-def detail + the scenarios that bind it, by expression.
  • step_catalog — reusable step vocabulary with extracted placeholders & allowed values (cucumber {type}, regex (a|b) enums).
  • list_tags — tag taxonomy with per-tag scenario counts.
  • project_dependencies — the implied project dependency graph (depends-on / depended-on-by), derived like the CLI map. Answers "what depends on the Party project?" over MCP.

Scope boundary (why no gap/hygiene tools)

Earlier drafts of this branch also added unbound_steps and coverage_gaps. Removed: SpecHygiene already owns step hygiene in both directions (unbound scenario steps + unused step definitions) authoritatively, so duplicating it here would add a second, map-staleness-prone source of truth. resolve_step retains the per-step authoring check; list_endpoints still exposes per-endpoint scenario reach for coverage questions.

Safety

  • All read-only. The 5 original tools are untouched.
  • resolve_step reuses the indexer's binder, so "would this bind?" matches runtime resolution.

Tests / eval

  • 235 → 247 tests, all green; the 235 originals unchanged.
  • Every tool has a durable test for its core behavior (incl. step_catalog enum extraction and resolve_step ranked near-miss suggestions).
  • Eval'd against a real ~25-project map: project_dependencies returned the 8 projects depending on OneFAT.API.Party; step_catalog pulled real allowed-values [Auto, Allianz, AllianzItaly, AllianzVoe].

Known limitations

  • resolve_step suggestions and step_catalog allowed-values are lexical/structural, not semantic.
  • Impact-style answers need a fresh v5 testatlas index (endpoints + full step-binding); a stale v4 map yields thin results.

Draft: prototype for review before any release / version bump.

Karzone added 6 commits July 29, 2026 19:07
…uthoring

resolve_step: match a Gherkin phrase to existing step definition(s) via the
real StepMatcher (regex/cucumber, keyword-agnostic) — exact/ambiguous/none,
with near-match suggestions when nothing binds. Reuse-first: stops agents
re-authoring steps that already exist.

unbound_steps: scenario steps with no matching definition, each with its
scenario/feature/location — the missing-glue worklist.

Read-only, existing tools untouched. +6 tests (241 total, all green).
…e multi-attribute duplicate

- suggestions now OR the phrase's salient tokens and rank by shared-term count,
  so a near-miss that substitutes a word still surfaces the closest steps
  (previously AND-of-tokens returned nothing on any substitution).
- a [Given]+[When]-on-one-method step now reads as one 'exact' step, not a
  false 'ambiguous'. +1 test (242 total, all green).
get_scenario: full scenario detail (feature, tags, kind, ordered steps) by name.
get_step_definition: step-def detail + the scenarios that bind it, by expression.
list_tags: tag taxonomy with per-tag scenario counts, most-used first.

Read-only; existing tools untouched. +3 tests (245 total, all green).
step_catalog: reusable step vocabulary with placeholder/allowed-value extraction
(cucumber {type}, regex (a|b) enums, free params) — compose scenarios from what exists.
coverage_gaps: untested endpoints (0 scenario reach) + unused step definitions.
project_dependencies: cross-project dependency graph (depends-on/depended-on-by),
derived like the CLI map — answers 'what depends on <project>?' over MCP.

Read-only; existing tools untouched. +3 tests (248 total, all green).
…tors

- durable test for the regex-alternation -> enum values path (was eval-only).
- coverage_gaps now returns totalEndpoints/totalStepDefinitions so '0 of 0'
  (no data indexed) is distinct from '0 of 50' (genuinely covered). +1 test (249).
@Karzone Karzone changed the title feat(mcp): resolve_step + unbound_steps for spec-driven authoring feat(mcp): 8 spec-driven authoring tools (resolve_step, unbound_steps, get_scenario/step_definition, step_catalog, coverage_gaps, list_tags, project_dependencies) Jul 29, 2026
…s step hygiene

Both detected step-gap/hygiene (unbound scenario steps, unused step definitions),
which SpecHygiene already covers authoritatively — avoid a second, map-staleness-prone
source of truth. resolve_step retains the per-step authoring check; list_endpoints
still exposes per-endpoint scenario reach. Net 6 new tools (11 total). 247 tests green.
@Karzone Karzone changed the title feat(mcp): 8 spec-driven authoring tools (resolve_step, unbound_steps, get_scenario/step_definition, step_catalog, coverage_gaps, list_tags, project_dependencies) feat(mcp): 6 spec-driven authoring tools (resolve_step, get_scenario/step_definition, step_catalog, list_tags, project_dependencies) Jul 29, 2026
@Karzone
Karzone marked this pull request as ready for review July 29, 2026 18:51
@Karzone
Karzone merged commit 9232cb2 into main Jul 29, 2026
1 check passed
@Karzone
Karzone deleted the feat/mcp-resolve-step-unbound-steps branch July 29, 2026 18:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant