Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
3d11d07
fix: update semantic conventions docs with correct attribute names an…
rogeriochaves Mar 11, 2026
c0da785
refactor: move SDK constants section after reference tables, add Pyth…
rogeriochaves Mar 11, 2026
e4bb007
feat: add skills-based onboarding pages with 4 paths
Mar 15, 2026
1be9a47
feat: add cross-link Tip callouts to feature pages pointing to skills…
Mar 15, 2026
64e7053
feat: add recipes page — domain-specific autoplayable skills catalog
Mar 16, 2026
70f3677
refactor(docs): workflow-based onboarding — coding assistant / chat a…
Mar 16, 2026
3bb76b4
fix(docs): code-prompts UX — smaller headers, direct copy blocks, rem…
Mar 16, 2026
c02c5e5
feat(docs): custom CopyPrompt component — compact copy button, Steps …
Mar 16, 2026
1044408
fix(docs): use template literals for prompts-data — fixes unterminate…
Mar 16, 2026
efa3e9b
fix: add React/useState import to CopyPrompt + error boundary
Mar 16, 2026
d40e63d
feat(docs): use CopyPrompt component on platform-prompts page
Mar 16, 2026
b2e45f1
refactor(docs): clean minimal layout — stacked copy buttons, single t…
Mar 16, 2026
2b93590
feat(docs): polished components — CopyLine, SkillInstall, rounded gap…
Mar 16, 2026
80e753c
fix: remove duplicate intro lines (already in frontmatter description)
Mar 16, 2026
99b4645
fix: bold 'All of the above' on level-up prompt
Mar 16, 2026
62348c2
feat(docs): add Copilot, ChatGPT, Claude Chat MCP instructions
Mar 16, 2026
0589aec
fix: add actual MCP server URL for ChatGPT and Claude Chat
Mar 16, 2026
4bb7831
fix: honest about ChatGPT/Claude Chat MCP — remote URL not available yet
Mar 16, 2026
487e50f
feat: add real MCP server URL for ChatGPT and Claude Chat
Mar 16, 2026
4de5257
fix: remove level-up from MCP section — too complex for MCP-only
Mar 16, 2026
8bd2f3b
refactor(docs): consistent layout across all skills pages
Mar 16, 2026
073c38f
fix(docs): use real compiled skill prompts for recipes, fix titles
Mar 16, 2026
a816323
fix: reorder recipes — 'What should I do next' first, rename debug title
Mar 16, 2026
1e7f3c0
fix: simpler compliance recipe title
Mar 16, 2026
3e702c0
fix: add star to 'What should I do next' recipe
Mar 16, 2026
d064be2
fix: render bold prefix in CopyPrompt via boldPrefix prop (markdown d…
Mar 17, 2026
1ca916a
fix: regenerate prompts-data with improved envelope — title first, .e…
Mar 17, 2026
ba32152
fix: regenerate prompts-data — user command as first line
Mar 17, 2026
460520f
docs: update MCP instructions — Bearer token auth, two modes (local/r…
Mar 17, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions agent-simulations/getting-started.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@
title: Getting Started
---

<Tip>
**Quick setup?** [Copy the scenarios prompt](/skills/code-prompts#add-scenario-tests) into your coding agent to add simulation tests automatically.
</Tip>

This guide will walk you through the basic setup required to run your first simulation and see the results in LangWatch.

For more in-depth information and advanced use cases, please refer to the official [`scenario` library documentation](https://github.com/langwatch/scenario).
Expand Down
4 changes: 4 additions & 0 deletions agent-simulations/introduction.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ sidebarTitle: Introduction
keywords: langwatch, agent simulations, agent testing, agent development, agent development, agent testing
---

<Tip>
**Quick setup?** [Copy the scenarios prompt](/skills/code-prompts#add-scenario-tests) into your coding agent to add simulation tests automatically.
</Tip>

# What are Agent Simulations?

Agent simulations are a powerful approach to testing AI agents that goes beyond traditional evaluation methods. Unlike static input-output testing, simulations test your agent's behavior in realistic, multi-turn conversations that mimic how real users would interact with your system.
Expand Down
2 changes: 1 addition & 1 deletion better-agents/overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Better Agents is a CLI tool and a set of standards for building **reliable, test
Use your preferred stack—Agno, Mastra, Vercel AI, Google ADK, or anything else. Better Agents doesn't replace your stack, it stabilizes it.

<Note>
Already have a project? Add evaluations, observability, and scenarios to your existing agent project. See the [Integration Guide](/integration/overview) to get started.
**Already have an agent?** You don't need Better Agents -- go to [LangWatch Skills](/skills/directory) to add tracing, evaluations, scenarios, and prompt versioning to your existing project.
</Note>

## Quick Start
Expand Down
4 changes: 4 additions & 0 deletions datasets/overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ sidebarTitle: Overview
description: Create and manage datasets in LangWatch to build evaluation sets for LLMs and structured AI agent testing.
---

<Tip>
**Let your agent set this up.** [Copy the evaluations prompt](/skills/code-prompts#set-up-evaluations) into your coding agent to get started automatically.
</Tip>

## Create datasets

LangWatch allows you to create and manage datasets, with a built-in excel-like interface for collaborating with your team.
Expand Down
35 changes: 32 additions & 3 deletions docs.json
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
},
"favicon": "/favicon.svg",
"banner": {
"content": "**[LangWatch MCP is live](https://langwatch.ai/docs/integration/mcp):** Your AI coding assistant can now build, version, and ship evals no platform context-switching required.",
"content": "**[LangWatch MCP is live](https://langwatch.ai/docs/integration/mcp):** Your AI coding assistant can now build, version, and ship evals \u2014 no platform context-switching required.",
"dismissible": true
},
"contextual": {
Expand Down Expand Up @@ -58,8 +58,17 @@
"group": "Get Started",
"pages": [
"introduction",
"better-agents/overview",
"integration/mcp"
{
"group": "LangWatch Skills",
"pages": [
"skills/directory",
"skills/code-prompts",
"skills/platform-prompts",
"skills/recipes"
]
},
"integration/mcp",
"better-agents/overview"
]
},
{
Expand Down Expand Up @@ -717,6 +726,26 @@
{
"source": "/hybrid-setup/sso-setup-langwatch",
"destination": "/self-hosting/sso-setup-langwatch"
},
{
"source": "/skills/overview",
"destination": "/skills/directory"
},
{
"source": "/skills/developers",
"destination": "/skills/code-prompts"
},
{
"source": "/skills/teams",
"destination": "/skills/platform-prompts"
},
{
"source": "/skills/platform",
"destination": "/skills/platform-prompts"
},
{
"source": "/skills/manual",
"destination": "/integration/quick-start"
}
]
}
4 changes: 4 additions & 0 deletions evaluations/evaluators/overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ sidebarTitle: Overview
description: Understand evaluators - the scoring functions that assess your LLM outputs for quality, safety, and correctness.
---

<Tip>
**Let your agent set this up.** [Copy the evaluations prompt](/skills/code-prompts#set-up-evaluations) into your coding agent to get started automatically.
</Tip>

Evaluators are scoring functions that assess the quality of your LLM's outputs. They're the building blocks for [experiments](/evaluations/experiments/overview), [online evaluation](/evaluations/online-evaluation/overview), and [guardrails](/evaluations/guardrails/overview).

## Choose Your Approach
Expand Down
4 changes: 4 additions & 0 deletions evaluations/experiments/overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ sidebarTitle: Overview
description: Run batch tests on your LLM applications to measure quality, compare configurations, and catch regressions before production.
---

<Tip>
**Let your agent set this up.** [Copy the evaluations prompt](/skills/code-prompts#set-up-evaluations) into your coding agent to get started automatically.
</Tip>

Experiments let you systematically test your LLM applications before deploying to production. Run your prompts, models, or agents against datasets and measure quality with evaluators.

## What is an Experiment?
Expand Down
4 changes: 4 additions & 0 deletions evaluations/experiments/sdk.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ sidebarTitle: Via SDK
description: Run experiments programmatically from notebooks or scripts to batch test your LLM applications.
---

<Tip>
**Let your agent set this up.** [Copy the evaluations prompt](/skills/code-prompts#set-up-evaluations) into your coding agent to get started automatically.
</Tip>

LangWatch makes it easy to run experiments from code.
Just add a few lines to start tracking your experiments.

Expand Down
4 changes: 4 additions & 0 deletions evaluations/guardrails/code-integration.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ sidebarTitle: Code Integration
description: Add guardrails to your LLM application to block harmful content in real-time.
---

<Tip>
**Let your agent set this up.** [Copy the evaluations prompt](/skills/code-prompts#set-up-evaluations) into your coding agent to get started automatically.
</Tip>

This guide shows how to integrate guardrails into your application using the LangWatch SDK. Guardrails run evaluators synchronously and return results you can act on immediately.

## Basic Usage
Expand Down
4 changes: 4 additions & 0 deletions evaluations/guardrails/overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ sidebarTitle: Overview
description: Block or modify harmful LLM responses in real-time to enforce safety and policy constraints.
---

<Tip>
**Let your agent set this up.** [Copy the evaluations prompt](/skills/code-prompts#set-up-evaluations) into your coding agent to get started automatically.
</Tip>

Guardrails are evaluators that run in real-time and **act** on the results - blocking, modifying, or rejecting responses that violate your safety or policy rules. Unlike [monitors](/evaluations/online-evaluation/overview) which only measure and alert, guardrails actively prevent harmful content from reaching users.

## Guardrails vs Monitors
Expand Down
4 changes: 4 additions & 0 deletions evaluations/online-evaluation/overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ sidebarTitle: Overview
description: Continuously score and monitor your LLM's production traffic for quality and safety with online evaluation.
---

<Tip>
**Let your agent set this up.** [Copy the evaluations prompt](/skills/code-prompts#set-up-evaluations) into your coding agent to get started automatically.
</Tip>

Online evaluation lets you continuously score your LLM's production traffic. Unlike [experiments](/evaluations/experiments/overview) which test before deployment, online evaluation monitors your live application to catch quality issues, detect regressions, and ensure safety.

<Info>
Expand Down
4 changes: 4 additions & 0 deletions evaluations/overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ sidebarTitle: Overview
description: Ensure quality and safety for your LLM applications with experiments, online evaluation, guardrails, and evaluators.
---

<Tip>
**Let your agent set this up.** [Copy the evaluations prompt](/skills/code-prompts#set-up-evaluations) into your coding agent to get started automatically.
</Tip>

LangWatch provides comprehensive evaluations tools for your LLM applications. Whether you're evaluating before deployment or monitoring in production, we have you covered.

## The Agent Evaluation Lifecycle
Expand Down
4 changes: 4 additions & 0 deletions integration/go/guide.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@ keywords: LangWatch, Go, Golang, SDK, integration, guide, setup, tracing, spans,

import LLMsTxtProtip from "/snippets/llms-txt-protip.mdx";

<Tip>
**Quick setup?** Instead of following these steps manually, [copy a prompt](/skills/code-prompts#instrument-my-code) into your coding agent and it will set this up for you automatically.
</Tip>

<div className="not-prose" style={{display: "flex", gap: "8px", padding: "0"}}>
<div>
<a href="https://github.com/langwatch/langwatch/tree/main/sdk-go" target="_blank">
Expand Down
62 changes: 51 additions & 11 deletions integration/mcp.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,14 @@ Go to your LangWatch project **Settings** page and copy your API key. The API ke
<Step title="Configure your MCP">

<Tabs>
<Tab title="Cursor">
1. Open Cursor Settings
2. Navigate to the **Tools and MCP** section in the sidebar
3. Add the LangWatch MCP server:
<Tab title="Claude Code">
Run this command to add the MCP server:

```bash
claude mcp add langwatch -- npx -y @langwatch/mcp-server --apiKey your-api-key-here
```

Or add it manually to your `~/.claude.json`:

```json
{
Expand All @@ -45,16 +49,30 @@ Go to your LangWatch project **Settings** page and copy your API key. The API ke
}
```

See the [Claude Code MCP documentation](https://code.claude.com/docs/en/mcp#plugin-provided-mcp-servers) for more details.
</Tab>

<Tab title="Claude Code">
Run this command to add the MCP server:
<Tab title="Copilot">
Add to `.vscode/mcp.json` in your project (or use **MCP: Add Server** from the Command Palette):

```bash
claude mcp add langwatch -- npx -y @langwatch/mcp-server --apiKey your-api-key-here
```json
{
"servers": {
"langwatch": {
"type": "stdio",
"command": "npx",
"args": ["-y", "@langwatch/mcp-server"],
"env": { "LANGWATCH_API_KEY": "your-api-key-here" }
}
}
}
```
</Tab>

Or add it manually to your `~/.claude.json`:
<Tab title="Cursor">
1. Open Cursor Settings
2. Navigate to the **Tools and MCP** section in the sidebar
3. Add the LangWatch MCP server:

```json
{
Expand All @@ -69,12 +87,27 @@ Or add it manually to your `~/.claude.json`:
}
}
```
</Tab>

See the [Claude Code MCP documentation](https://code.claude.com/docs/en/mcp#plugin-provided-mcp-servers) for more details.
<Tab title="ChatGPT">
1. Go to **Settings → Connectors**
2. Click **Add connector**
3. Enter the server URL: `https://mcp.langwatch.ai/sse`
4. For authentication, select **Bearer Token** and enter your LangWatch API key (get one at [app.langwatch.ai/authorize](https://app.langwatch.ai/authorize))

*Requires a Plus or Team plan.*
</Tab>

<Tab title="Other Editors">
<Tab title="Claude Chat">
1. Go to **Settings → Connectors**
2. Click **Add custom connector**
3. Enter the server URL: `https://mcp.langwatch.ai/sse`
4. Click **Advanced settings**, select **Bearer Token** auth, and enter your LangWatch API key (get one at [app.langwatch.ai/authorize](https://app.langwatch.ai/authorize))

*Requires a Pro or Max plan.*
</Tab>

<Tab title="Other">
For other MCP-compatible editors, add the following configuration to your MCP settings file:

```json
Expand Down Expand Up @@ -109,6 +142,13 @@ Open your AI assistant chat (e.g., `Cmd/Ctrl + I` in Cursor, or `Cmd/Ctrl + Shif
| `LANGWATCH_API_KEY` | `--apiKey` | API key for authentication |
| `LANGWATCH_ENDPOINT` | `--endpoint` | API endpoint (default: `https://app.langwatch.ai`) |

### Two Modes

The MCP server runs in two modes:

- **Local (stdio)**: Default. Runs as a subprocess of your coding assistant (Claude Code, Copilot, Cursor). API key set via `--apiKey` flag or `LANGWATCH_API_KEY` env var.
- **Remote (HTTP/SSE)**: For web-based assistants (ChatGPT, Claude Chat). Hosted at `https://mcp.langwatch.ai`. API key sent as `Authorization: Bearer <key>` per session — each user brings their own key.

## Usage Examples

### Write Agent Tests with Scenario
Expand Down
4 changes: 4 additions & 0 deletions integration/python/guide.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@ keywords: LangWatch, Python, SDK, integration, guide, setup, tracing, spans, tra

import LLMsTxtProtip from "/snippets/llms-txt-protip.mdx";

<Tip>
**Quick setup?** Instead of following these steps manually, [copy a prompt](/skills/code-prompts#instrument-my-code) into your coding agent and it will set this up for you automatically.
</Tip>

<div className="not-prose" style={{display: "flex", gap: "8px", padding: "0"}}>
<div>
<a href="https://github.com/langwatch/langwatch/tree/main/python-sdk" target="_blank">
Expand Down
4 changes: 4 additions & 0 deletions integration/python/integrations/agno.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,10 @@ description: Instrument Agno agents with LangWatch’s Python SDK to send traces
keywords: agno, openinference, langwatch, python, tracing, observability
---

<Tip>
**Quick setup?** Instead of following these steps manually, [copy a prompt](/skills/code-prompts#instrument-my-code) into your coding agent and it will set this up for you automatically.
</Tip>

LangWatch integrates with Agno through OpenInference instrumentation to capture traces from your Agno agents automatically.

## Installation
Expand Down
4 changes: 4 additions & 0 deletions integration/python/integrations/langchain.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,10 @@ icon: python
keywords: langchain, instrumentation, callback, langwatch, python, tracing
---

<Tip>
**Quick setup?** Instead of following these steps manually, [copy a prompt](/skills/code-prompts#instrument-my-code) into your coding agent and it will set this up for you automatically.
</Tip>

LangWatch integrates with Langchain to provide detailed observability into your chains, agents, LLM calls, and tool usage.

## Installation
Expand Down
4 changes: 4 additions & 0 deletions integration/python/integrations/langgraph.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,10 @@ icon: python
keywords: langgraph, instrumentation, callback, langwatch, python, tracing
---

<Tip>
**Quick setup?** Instead of following these steps manually, [copy a prompt](/skills/code-prompts#instrument-my-code) into your coding agent and it will set this up for you automatically.
</Tip>

LangWatch integrates with LangGraph to provide detailed observability into your graph-based agents, LLM calls, and tool usage.

## Installation
Expand Down
4 changes: 4 additions & 0 deletions integration/python/integrations/open-ai.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,10 @@ icon: python
keywords: openai, instrumentation, autotrack, langwatch, python
---

<Tip>
**Quick setup?** Instead of following these steps manually, [copy a prompt](/skills/code-prompts#instrument-my-code) into your coding agent and it will set this up for you automatically.
</Tip>

LangWatch integrates with OpenAI to automatically capture detailed information about your LLM calls.

## Installation
Expand Down
4 changes: 4 additions & 0 deletions integration/quick-start.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,10 @@ title: Quick Start
mode: "wide"
---

<Tip>
**Quick setup?** Instead of following these steps manually, [copy a prompt](/skills/code-prompts#instrument-my-code) into your coding agent and it will set this up for you automatically.
</Tip>

LangWatch helps you understand every user interaction (**Thread**), each individual AI task (**Trace**), and all the underlying steps (**Span**) involved. We've made getting started super smooth.

Let's get cracking.
Expand Down
4 changes: 4 additions & 0 deletions integration/typescript/guide.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@ keywords: langwatch, typescript, sdk, guide, observability, tracing, logging, da

import LLMsTxtProtip from "/snippets/llms-txt-protip.mdx";

<Tip>
**Quick setup?** Instead of following these steps manually, [copy a prompt](/skills/code-prompts#instrument-my-code) into your coding agent and it will set this up for you automatically.
</Tip>

<div className="not-prose" style={{display: "flex", gap: "8px", padding: "0"}}>
<div>
<a href="https://github.com/langwatch/langwatch/tree/main/typescript-sdk" target="_blank">
Expand Down
4 changes: 4 additions & 0 deletions integration/typescript/integrations/langchain.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,10 @@ icon: square-js
keywords: langchain, instrumentation, callback, langwatch, typescript, tracing
---

<Tip>
**Quick setup?** Instead of following these steps manually, [copy a prompt](/skills/code-prompts#instrument-my-code) into your coding agent and it will set this up for you automatically.
</Tip>

LangWatch integrates with Langchain to provide detailed observability into your chains, agents, LLM calls, and tool usage.

## Installation
Expand Down
4 changes: 4 additions & 0 deletions integration/typescript/integrations/mastra.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,10 @@ sidebarTitle: Mastra
keywords: mastra, langwatch, tracing, observability, typescript, agent framework, ai agents
---

<Tip>
**Quick setup?** Instead of following these steps manually, [copy a prompt](/skills/code-prompts#instrument-my-code) into your coding agent and it will set this up for you automatically.
</Tip>

LangWatch integrates with Mastra through OpenTelemetry to capture traces from your Mastra agents automatically.

## Installation
Expand Down
4 changes: 4 additions & 0 deletions integration/typescript/integrations/vercel-ai-sdk.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@ keywords: vercel ai sdk, langwatch, tracing, observability, vercel, ai, sdk

import TypeScriptIntro from "/snippets/typescript-intro.mdx";

<Tip>
**Quick setup?** Instead of following these steps manually, [copy a prompt](/skills/code-prompts#instrument-my-code) into your coding agent and it will set this up for you automatically.
</Tip>

<TypeScriptIntro />

## Installation
Expand Down
Loading
Loading