Skip to content

Allow opt-in Copilot macro tools in bypass mode - #3216

Open
George Ng (GeorgeNgMsft) wants to merge 2 commits into
mainfrom
fix/copilot-bypass-macro-tools
Open

George Ng (GeorgeNgMsft) wants to merge 2 commits into
mainfrom
fix/copilot-bypass-macro-tools

Conversation

@GeorgeNgMsft

@GeorgeNgMsft George Ng (GeorgeNgMsft) commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

What changes

Copilot already has a run_macro MCP tool, but its adapter rejects every call in bypass mode. This adds a default-off opt-in for explicit macro discovery and checked execution while ordinary prompts stay with Copilot.

Set TYPEAGENT_BYPASS_MACRO_TOOLS=true before starting Copilot, or save "bypassMacroTools": true alongside "mode": "bypass" in the plugin config. The environment overrides the file setting. Invalid values are rejected rather than enabling access.

The opt-in allows list_macros, search_macros, inspect_macro, get_macro_requirements, run_macro, and get_macro_run. Validation, approval, disablement, and deletion remain blocked in bypass.

Both status surfaces distinguish macro availability from prompt routing. The README documents setup and the limits.

What stays unchanged

Execution uses the existing local owner and macro manager, including current approved-version checks, backend authorization, native session/workspace binding, cancellation, and durable run results. A disabled owner stays disabled.

This does not enable automatic recipe matching, TypeAgent delegation, workspace tools, learning, or additional tool permissions. Unsupported tools and backend failures do not fall back to a planner or another provider. Capturing a trace still does not make it an executable macro.

Calls aborted before or during owner connection now stop before dispatch; cancellation of an active run still forwards its preallocated run ID.

This branch starts directly from main and is independent of the capture fix (#3214) and runtime discovery fix (#3215).

Validation

  • Dependency-aware build and final plugin build passed.
  • Full plugin suite: 565 Jest tests, 3 extension-layout tests, and 12 launcher tests passed.
  • Nine new integration cases passed again after the final input-discovery/latest-version assertions.
  • Pinned Prettier check passed.
  • All four PR gates passed against origin/main on the committed change: lint, complexity, circular dependencies, and debt.
  • Direct adversarial review covered default-off behavior, malformed settings, all ten tool admissions, unchanged prompt routing, owner-off behavior, backend rejection, and cancellation.

The follow-up adversarial review did not confirm a new blocking product defect. It closed a verification gap: the original happy-path tests mocked execution, while the real-owner test only rejected a missing macro.

The new integration suite launches the actual bundled macro MCP server over stdio, connects to the real local owner/broker, executes an approved compiled program with the real RecipeExecutor/native host, and reads the durable result. Both environment and saved-config opt-ins succeed. Default-off/false/invalid flags, lifecycle operations, draft and disabled/stale versions, invalid inputs, and an inactive owner fail without native dispatch. A backend permission denial remains denied with exactly one call and no fallback.

Only the native SDK/tool provider is a controlled read-tool fixture; this is not live native Copilot/backend end-to-end execution of a user's saved macro. The synthetic catalog is isolated, and no real learning preference or permission setting is changed. This PR does not claim additional tool replay support.

Keep prompt routing and macro management disabled while allowing explicit discovery and checked execution through the existing local owner. Preserve backend authorization, reject invalid opt-ins, and stop aborted calls before dispatch.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 0040b8ff-2701-4fb7-821f-760e05fe6b4b
Add approved-program integration coverage using the actual bundled stdio server, local owner, compiled executor, and native host with an isolated SDK fixture. Verify opt-in precedence, durable output, version checks, lifecycle rejection, owner shutdown, and backend denial without retry.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 0040b8ff-2701-4fb7-821f-760e05fe6b4b
@GeorgeNgMsft

Copy link
Copy Markdown
Contributor Author

Adversarial review follow-up

No new blocking product defect was confirmed. The main gap was proof: the original successful-run tests mocked the owner call, while the real-owner test only rejected an unknown macro.

Commit 90cfa6a adds nine integration cases using the actual bundled macro MCP stdio server, real local owner/broker, real compiled RecipeExecutor/native host, and an isolated synthetic approved catalog. The positive case discovers the macro, gets its input requirements, executes the current approved version without supplying a version or inspecting first, checks the actual fixture-read result and beta.txt dispatch, and reads the persisted run through get_macro_run.

The tests also verify saved-config opt-in, default-off/false/invalid flags, blocked lifecycle operations, draft and disabled/stale versions, invalid inputs, inactive owner, and an authoritative backend denial with exactly one call and no fallback.

Validation:

  • Plugin build passed.
  • Full suite: 565 Jest tests, 3 layout tests, 12 launcher tests passed.
  • Final integration rerun: 9/9 passed.
  • Prettier and all four committed-change ratchets passed (lint, complexity, circular dependencies, debt).

Boundary: the native SDK/tool provider is a controlled read-tool fixture. This proves the production MCP/owner/program execution path, not live Copilot execution against an actual backend or automatic natural-language macro selection. No real learning preferences or permission settings were changed. CI is being rerun for this follow-up commit.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants