Skip to content

chore: sync microsoft-365-agents-toolkit plugin and skill #1999

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
qfai wants to merge 1 commit into
base: staged
Choose a base branch
from
Open

chore: sync microsoft-365-agents-toolkit plugin and skill #1999

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 55 additions & 0 deletions plugins/microsoft-365-agents-toolkit/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# M365 Agents Toolkit

Toolkit for building Microsoft 365 Copilot declarative agents.

## Installation

### Via GitHub Copilot CLI Plugin Marketplace

```bash
/plugin install microsoft-365-agents-toolkit@work-iq
```

## Usage

```
# Develop an agent
"Scaffold a new declarative agent for HR FAQ"

# Configure capabilities
"Add web search to my agent"

# Deploy
"Deploy my agent with ATK"

# Create evals
"Create an eval suite for my agent based on it's capabilities."

# Run evals
"Run my evals for the agent"

# Analyze and improve
"Analyze the evaluation failures by root cause, and recommend targeted agent instruction changes"

# Regression check after agent changes
"I changed my agent instructions. Re-run the evals with stable concurrency and compare the new results to .evals\baseline.json"
```

The evaluator skill uses the public preview M365 Copilot eval CLI through package-scoped `npx`. Learn more about the preview, docs, issues, and feedback channels in the public [m365-copilot-eval repository](https://github.com/microsoft/m365-copilot-eval).

```powershell
npx -y --package @microsoft/m365-copilot-eval@latest runevals --prompts-file evals\evals.json --output .evals\latest.json
```

## Skills

| Skill | What It Does |
|-------|-------------|
| [**install-atk**](./skills/install-atk/SKILL.md) | Install or update the ATK CLI and VS Code extension |
| [**declarative-agent-developer**](./skills/declarative-agent-developer/SKILL.md) | Scaffolding, JSON manifest authoring, capability configuration, security patterns, deployment via ATK CLI |
| [**ui-widget-developer**](./skills/ui-widget-developer/SKILL.md) | Build MCP servers with OpenAI Apps SDK widget rendering for Copilot Chat |
| [**m365-agent-evaluator**](./skills/m365-agent-evaluator/SKILL.md) | Generate, run, and analyze evaluation suites for M365 Copilot declarative agents |

## License

See the root [LICENSE](../../LICENSE) file.
114 changes: 114 additions & 0 deletions skills/microsoft-365-agents-toolkit/SKILL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
---
name: microsoft-365-agents-toolkit
description: "Builds, tests, and deploys Microsoft 365 apps and agents for Teams and Copilot. Includes sub-skills for project creation, local testing, cloud deployment, troubleshooting, and Slack-to-Teams migration. USE FOR: Teams agent, bot, tab, message extension, Declarative Agents, Custom Engine Agents, local testing, Agents Playground, Azure resource provision, remote deployment, Slack to Teams migration, cross-platform bot development, Block Kit to Adaptive Cards conversion. DO NOT USE FOR: general web development, non-bot/non-Teams projects."
---

# Microsoft 365 Agents Toolkit Skill

Build Microsoft 365 agents and Teams apps using the ATK CLI.

## AI Behavior Guidelines

1. **Testing Strategy:** Recommend Agents Playground first (faster, no M365 needed). Use Teams workflow only if user explicitly requests it.

2. **Environment Variables:** NEVER hardcode secrets or make up placeholder values. Always ask users for real values.

3. **Error Handling:** Read error messages carefully. Check `env/.env.local`, `.localConfigs`, and `atk auth list`. Common pitfalls:
- **`AADSTS7000229`** → `aadApp/create` missing `generateServicePrincipal: true` in YAML — add it and re-provision
- **Missing `TENANT_ID`** in `.localConfigs` → SDK uses wrong token authority → 401 from Bot Connector
- **401 persists after auth fix** → devtunnel URL may be blacklisted — create a fresh tunnel
- See [troubleshoot/troubleshoot.md](troubleshoot/troubleshoot.md) for full diagnostic steps

4. **Long-Running Commands — WAIT for completion:**
- `atk new`, `atk provision`, `atk deploy` can take several minutes
- Always wait for completion before running the next step (timeout 120000ms+)

5. **Local Service Startup — Hangs terminal (expected):**
- `npm run dev`, `npm start`, `python app.py`, `devtunnel host`, etc. will hang — the process keeps running indefinitely
- ALWAYS run as a background process (`isBackground=true`) — NEVER use `isBackground=false` for these commands
- Do NOT wait for it to "finish" — verify startup by checking output for "listening on port" or tunnel URL
- If errors appear, read logs, diagnose, fix, restart
- Use a **NEW terminal** to launch Agents Playground or open Teams sideloading URL

6. **Monitor App Logs:** Periodically check background terminal output for runtime errors. If the app crashes, read the error, fix the root cause, and restart.

7. **Telemetry Tagging:** Before running any `atk` CLI commands, set the session environment variable so all CLI invocations are tagged as skill-initiated:
```bash
export ATK_CLI_SKILL=true
```
Run this once at the start of the session. All subsequent `atk` commands in the same terminal will inherit it.

## ATK CLI Setup

```bash
atk --version # Must be > 1.1.5-beta
```

If ATK is not found or version is too old:
```bash
npm i -g @microsoft/m365agentstoolkit-cli@beta
```

## CLI Global Options

| Option | Meaning | Recommendation |
| --- | --- | --- |
| `-i` | Interactive mode | Always use `-i false` in automation to avoid hanging |
| `-f` | Project folder | Default to be current directory, used when specifying a custom folder. When scaffolding a new project, this is the parent folder where the project folder will be created under. |
| `-h` | Command help | Use `atk <command> -h` for quick syntax checks |

## Sub-Skills

| Sub-Skill | When to Use | Reference |
|-----------|-------------|-----------|
| **create-project** | Scaffold new project from template, choose template, `atk new` | [create-project/create-project.md](create-project/create-project.md) |
| **test-playground** | Test locally with Agents Playground, `agentsplayground`, quick testing | [test-playground/test-playground.md](test-playground/test-playground.md) |
| **test-teams** | Run on Teams, devtunnel, sideload, Teams testing, test in Copilot | [test-teams/test-teams.md](test-teams/test-teams.md) |
| **provision-deploy** | Provision Azure resources, deploy to cloud, `atk provision`, `atk deploy` | [provision-deploy/provision-deploy.md](provision-deploy/provision-deploy.md) |
| **troubleshoot** | Fix errors, 401, port conflicts, YAML errors, stale bots | [troubleshoot/troubleshoot.md](troubleshoot/troubleshoot.md) |
| **slack-to-teams** | Migrate Slack bot to Teams, cross-platform bridging, Block Kit to Adaptive Cards | [slack-to-teams/SKILL.md](slack-to-teams/SKILL.md) |

> **MANDATORY:** Before executing any workflow, read the corresponding sub-skill document.

## Shared References

- [manifest-and-yaml.md](toolkit/manifest-and-yaml.md) — Project files, YAML config, env vars, .localConfigs flow
- [commands.md](toolkit/commands.md) — ATK CLI commands: package, validate, share, collaborate
- [templates.md](toolkit/templates.md) — Complete template catalog with language support
- [experts/](experts/index.md) — 100+ micro-expert files: Teams SDK, Slack SDK, cross-platform bridging, deploy, AI models, security, language conversion
- [docs/](docs/README.md) — Platform comparison guides: UI, messaging, identity, infrastructure, feature gaps

## Workflow Chains

Match user intent to the smallest valid workflow.

| User Intent | Workflow (read in order) |
|---|---|
| Build new app from scratch | create-project → test-playground |
| Test existing project locally | test-playground (recommended) or test-teams |
| Deploy to Azure | provision-deploy |
| Fix broken bot | troubleshoot → re-test |
| Migrate Slack bot to Teams | slack-to-teams |

> **MANDATORY:** Before executing any slack-to-teams workflow, read [slack-to-teams/SKILL.md](slack-to-teams/SKILL.md) first. The sub-skill contains a routed expert system with 100+ micro-expert files for cross-platform bot development.

## ATK Project Context Resolution

Resolve config values only when missing. If a value is already known in the session, reuse it.

### Step 1: Detect ATK Project

If `m365agentstoolkit*.yml` exists in the current folder, treat it as an ATK project and parse configuration.

### Step 2: Resolve Common Configuration

Resolve variables referenced in `m365agentstoolkit*.yml`. Common variables:
AZURE_OPENAI_API_KEY
Comment on lines +99 to +106
AZURE_OPENAI_ENDPOINT
AZURE_OPENAI_DEPLOYMENT_NAME

### Step 3: Collect Missing Values

If required values are missing, ask the user for only the missing ones.

Refer to [manifest-and-yaml.md](toolkit/manifest-and-yaml.md) for full config-file details.
134 changes: 134 additions & 0 deletions skills/microsoft-365-agents-toolkit/create-project/create-project.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
# Create Project

Scaffold a new Microsoft 365 agent or Teams app from an ATK template.

## Template Selection Guide

| User Wants | Capability |
|------------|------------|
| Extend M365 Copilot with custom instructions | `declarative-agent` |
| Declarative Agent with new API | `declarative-agent-action` |
| Declarative Agent with new API (Bearer Token) | `declarative-agent-action-bearer` |
| Declarative Agent with new API (OAuth) | `declarative-agent-action-oauth` |
| Declarative Agent with existing OpenAPI spec | `declarative-agent-action-from-existing-api` |
| Connect MCP Server to Copilot | `declarative-agent-with-action-from-mcp` |
| Declarative Agent with Copilot Connector | `declarative-agent-with-graph-connector` |
| Declarative Agent for MetaOS | `declarative-agent-meta-os-new-project` |
| Declarative Agent from TypeSpec | `declarative-agent-typespec` |
| Agent with custom LLM (Azure OpenAI, etc.) | `basic-custom-engine-agent` |
| Weather forecast agent | `weather-agent` |
| Agent using Azure AI Foundry | `foundry-agent-to-m365` |
| Teams chatbot with AI | `teams-agent` |
| Teams bot with RAG/knowledge base | `teams-agent-rag-customize` |
| Teams Agent with Azure AI Search | `teams-agent-rag-azure-ai-search` |
| Teams Agent with Custom API | `teams-agent-rag-custom-api` |
| Teams Collaborator Agent | `teams-collaborator-agent` |
| Simple Teams echo bot | `bot` |
| Teams tab app | `tab` |
| Teams message extension | `message-extension` |
| Copilot Connector | `copilot-connector` |

See [../toolkit/templates.md](../toolkit/templates.md) for the complete template catalog with language support and descriptions.

## Creating Projects

Create templates in the current directory with one generic flow:

```bash
# 1) Scaffold into a temporary parent folder
atk new -c <template-id> -n <project-name> -f /tmp -l <language> -i false

# 2) Move generated files from the scaffold subfolder into current directory
mv /tmp/<project-name>/. .

# 3) Remove the empty scaffold folder
rmdir /tmp/<project-name>
```

Common examples:

```bash
# Declarative Agent (no -l needed)
atk new -c declarative-agent -n my-agent -f /tmp -i false

# Declarative Agent with new API
atk new -c declarative-agent-action -l typescript -n my-api-agent -f /tmp -i false

# Declarative Agent with existing OpenAPI spec
atk new -c declarative-agent-action-from-existing-api -n my-agent -a <openapi-spec-url-or-path> -o "GET /repairs" -o "POST /repairs" -f /tmp -i false

# Custom Engine Agent
atk new -c basic-custom-engine-agent -l typescript -n my-cea -f /tmp -i false

# Teams Agent with RAG
atk new -c teams-agent-rag-customize -l typescript -n my-rag-agent -f /tmp -i false
```

PowerShell equivalent:

```powershell
# 1) Scaffold into temporary folder
atk new -c <template-id> -n <project-name> -f $env:TEMP -l <language> -i false

# 2) Move files into current directory
Move-Item "$env:TEMP\<project-name>\*" .
Move-Item "$env:TEMP\<project-name>\.*" . -ErrorAction SilentlyContinue

# 3) Remove scaffold folder
Remove-Item "$env:TEMP\<project-name>" -Force
```

## Creating from Samples

```bash
atk new sample <sample-id>
```

To place sample files in current directory, scaffold first and then move files from the sample output folder into `.` using the same move pattern as above.

| Sample | Sample ID (`atk new sample <sample-id>`) | Tags |
| ------------------------------------------------- | ---------------------------------------- | ----------------------------------------------------------------------------------------- |
| Langchain Agent with Agent365 SDK in NodeJS | `agent365-langchain-nodejs` | Agent365, TS |
| Agent Framework Agent with Agent365 SDK in Python | `agent365-agentframework-python` | Agent365, Python |
| OpenAI Agent with Agent365 SDK in Python | `agent365-openai-python` | Agent365, Python |
| Claude Agent with Agent365 SDK in NodeJS | `agent365-claude-nodejs` | Agent365, TS |
| Tab App with Azure Backend | `hello-world-tab-with-backend` | Tab, TS, Azure Functions, Dev Proxy |
| Bot App with SSO Enabled | `bot-sso` | Bot, TS, Adaptive Cards, SSO |
| Team Central Dashboard | `team-central-dashboard` | Tab, TS, Azure Functions, SSO |
| Copilot connector App | `copilot-connector-app` | Tab, Azure Functions, TS, SSO, Copilot connector |
| Teams Conversation Bot using Python | `bot-conversation-python` | Python, Bot, Bot Framework |
| Teams Messaging Extensions Search using Python | `msgext-search-python` | Python, Message extension, Bot Framework |
| Travel Agent | `travel-agent` | C#, Custom Engine Agent, M365 Copilot Retrieval API, Agents SDK, Agent Framework |
| Coffee Agent | `coffee-agent` | TS, Custom Engine Agent, Adaptive Cards, Microsoft Teams SDK |
| Data Analyst Agent v2 | `data-analyst-agent-v2` | TS, Custom Engine Agent, Data Visualization, Adaptive Cards, LLM SQL, Microsoft Teams SDK |

List all samples with `atk list samples`.

## Notes

- `declarative-agent` does NOT require `-l` language flag
- `declarative-agent-action-from-existing-api` requires `-a` (OpenAPI spec) and `-o` (operation IDs like `"GET /path"`)
- Always use `-i false` for non-interactive scripted creation
- `atk new` can take several minutes — wait for completion (timeout 120000ms+)
- If template/sample already matches the requirement, do not run dependency install by default; continue only when user asks for next steps

## After Scaffolding

Once the project is created:
- To test locally → see [../test-playground/test-playground.md](../test-playground/test-playground.md)
- To understand project files → see [../toolkit/manifest-and-yaml.md](../toolkit/manifest-and-yaml.md)

## Expert Deep Dives

> **Applies to: code-based Teams bots/agents only** (templates: `bot`, `teams-agent*`, `basic-custom-engine-agent`, `weather-agent`, `coffee-agent`, `bot-sso`, `msgext-*`, `tab*`).
>
> Does **not** apply to declarative agents, API plugins, Copilot connectors, or `declarative-agent-*` / `copilot-connector` templates — those have no source code to scaffold against. For those, follow the in-template instructions and the [Microsoft 365 Copilot extensibility docs](https://learn.microsoft.com/microsoft-365-copilot/extensibility/) directly.

For deeper guidance on what `atk new` produces and how to extend it, consult the Teams expert micro-files:

| Topic | Expert |
|---|---|
| Project file layout, `package.json`, `tsconfig.json`, npm scripts, `appPackage/` | [../experts/teams/project.scaffold-files-ts.md](../experts/teams/project.scaffold-files-ts.md) |
| `App` constructor, plugins, credentials, runtime initialization | [../experts/teams/runtime.app-init-ts.md](../experts/teams/runtime.app-init-ts.md) |
| Teams app manifest schema, scopes, bots/composeExtensions/staticTabs | [../experts/teams/runtime.manifest-ts.md](../experts/teams/runtime.manifest-ts.md) |
| Routing handlers (`app.on('message')`, activity types, invokes) | [../experts/teams/runtime.routing-handlers-ts.md](../experts/teams/runtime.routing-handlers-ts.md) |
46 changes: 46 additions & 0 deletions skills/microsoft-365-agents-toolkit/docs/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# Slack vs Teams: Platform Differences & Bridging Strategies

A practical guide for developers adding cross-platform support to an existing bot. Each document covers a category of differences, explains why they matter, and provides concrete mitigation strategies with effort estimates.

## Documents

| Document | What It Covers |
|---|---|
| [**Feature Gaps**](feature-gaps.md) | **Complete inventory of every RED and YELLOW gap with mitigations in both directions** |
| [**Workflows**](workflows.md) | **Message-native workflow scenarios: standup, PTO, equipment, account health, break management, incidents** |
| [Messaging & Commands](messaging-and-commands.md) | Messages, slash commands, events, threading, @mentions |
| [UI Components](ui-components.md) | Block Kit vs Adaptive Cards, modals vs dialogs, App Home vs personal tabs |
| [Interactive Responses](interactive-responses.md) | Ephemeral messages, button actions, message updates, confirmation dialogs |
| [Identity & Auth](identity-and-auth.md) | User IDs, OAuth, signing/verification, tokens |
| [Files & Links](files-and-links.md) | File upload/download, link unfurling/previews |
| [Middleware & Handler Patterns](middleware-and-handlers.md) | Middleware chains, ack(), handler registration, error handling |
| [Advanced Features](advanced-features.md) | Scheduling, workflows, shortcuts, channel ops, reactions, distribution |
| [Infrastructure](infrastructure.md) | Transport, compute, storage, secrets, observability |
| [**Eval Harness**](../evals/README.md) | Automated testing for expert routing, completeness, and code patterns |

## Eval Harness

The [`evals/`](../evals/) directory contains an automated test harness for the expert system. It validates three dimensions:

- **Routing** — 51 test cases across all 7 domains verify queries route to the correct domain, clusters, and expert files
- **Completeness** — 9 test cases check experts cover all required concepts for their domain
- **Patterns** — 294 TypeScript code blocks across all experts are compiled in-memory to catch syntax errors

Pattern evals are fully deterministic (no API key needed). Routing and completeness evals use an LLM judge (OpenAI, Anthropic, or Azure OpenAI). See [`evals/README.md`](../evals/README.md) for setup and usage.

## How to Read These Docs

Each difference follows this format:

- **What's different** — the concrete behavioral gap
- **Impact** — what breaks or degrades if you ignore it
- **Mitigation** — one or more strategies ranked by effort and fidelity
- **Effort** — rough hours to implement

### Difficulty Ratings

| Rating | Meaning |
|---|---|
| GREEN | Direct mapping exists. Mechanical conversion, minimal design decisions. |
| YELLOW | Mapping exists but requires design decisions or trade-offs. |
| RED | Platform gap — no equivalent exists. Requires redesign or custom workaround. |
Loading
Loading