docs: document guideline generation modes and add end-to-end tutorial

.gitignore

@@ -22,3 +22,4 @@ event.json
 site/
 example-guidelines
 .codex
+notes/

docs/guides/configuration.md

@@ -45,6 +45,10 @@ All configuration variables are prefixed with `EVOLVE_`.
 |----------|-------------------------------------------------------------------------------|------------------------------------------|
 | `EVOLVE_BACKEND` | Backend provider (`milvus`, `filesystem`, or `postgres`)                      | `milvus`                                 |
 | `EVOLVE_NAMESPACE_ID` | Namespace ID for isolation                                                    | `evolve`                                 |
+| `EVOLVE_GUIDELINES_MODE` | Guideline generation pipeline: `regular`, `consistency`, or `both` — see [Enabling Guidelines](guidelines.md) | `regular` |
+| `EVOLVE_HIGH_UNCERTAINTY_THRESHOLD` | Consistency mode: steps scoring above this are treated as high-uncertainty — see [Enabling Guidelines](guidelines.md#configuring-consistency-guideline-generation) | `0.2` |
+| `EVOLVE_LOW_UNCERTAINTY_THRESHOLD` | Consistency mode: steps scoring below this are treated as stable — see [Enabling Guidelines](guidelines.md#configuring-consistency-guideline-generation) | `0.1` |
+| `EVOLVE_SKIP_ON_NO_UNCERTAINTY` | Consistency mode: skip guideline generation if no step exceeds the uncertainty threshold — see [Enabling Guidelines](guidelines.md#configuring-consistency-guideline-generation) | `true` |
 | `EVOLVE_GUIDELINES_MODEL` | Model for guideline generation only | `EVOLVE_MODEL_NAME` -> `gpt-4o` |
 | `EVOLVE_CONFLICT_RESOLUTION_MODEL` | Model for conflict resolution only | `EVOLVE_MODEL_NAME` -> `gpt-4o` |
 | `EVOLVE_FACT_EXTRACTION_MODEL` | Model for fact extraction only | `EVOLVE_MODEL_NAME` -> `gpt-4o` |

docs/guides/extract-trajectories.md

@@ -2,6 +2,8 @@
 
 A Python utility at `scripts/extract_trajectories.py` for extracting agent trajectories from Arize Phoenix traces and converting them to OpenAI chat completion message format.
 
+This is a standalone export/debugging tool — it prints trajectories to stdout or a file and does not talk to Evolve or generate guidelines. To pull trajectories into Evolve *and* generate guidelines from them, use [Phoenix Sync](phoenix-sync.md) instead.
+
 ## Features
 
 - Fetches spans from Phoenix's REST API with pagination support

docs/guides/guidelines.md

@@ -0,0 +1,71 @@
+# Enabling Guidelines
+
+Guidelines are short, actionable recommendations Evolve extracts from agent conversations ("trajectories") and stores as `guideline` entities. This guide covers how guideline generation works in **full Evolve (MCP server / CLI)** and how to choose between the two available generation methods, **regular** and **consistency**.
+
+> This guide applies to full Evolve. It does not apply to [Evolve Lite](../integrations/claude/evolve-lite.md), where guideline extraction happens entirely inside the host agent's own reasoning (a prompt-driven skill) rather than through the LLM pipeline described here.
+
+## Two ways trajectories reach the guideline pipeline
+
+| Entry point | When it runs |
+|---|---|
+| `save_trajectory` MCP tool | Called directly by an MCP client (e.g. Claude Desktop, Claude Code) at the end of a conversation |
+| `evolve sync phoenix` CLI | Pulls previously-traced conversations out of Arize Phoenix in a batch — see [Phoenix Sync](phoenix-sync.md), which requires traces already flowing in via [Low-Code Tracing](low-code-tracing.md) |
+
+Both entry points funnel into the same underlying pipeline and respect the same `EVOLVE_GUIDELINES_MODE` setting below.
+
+## Choosing a guideline generation mode
+
+Set `EVOLVE_GUIDELINES_MODE` (or pass `--guidelines-mode` to `evolve sync phoenix`) to control which pipeline(s) run:
+
+| Mode | Optimizes for | What it does |
+|---|---|---|
+| `regular` (default) | Correctness on a single run | Single LLM pass over the trajectory; produces one guideline set. |
+| `consistency` | Reliability across repeated runs | Resampling pass to score agent decision steps in the trajectory for consistency followed by a focused LLM pass to produce guidelines for inconsistent steps.  |
+| `both` | Both | Runs both pipelines and stores both sets of guidelines side by side. |
+
+```bash
+# Regular guidelines (default) — no change needed
+export EVOLVE_GUIDELINES_MODE=regular
+
+# Consistency guidelines only
+export EVOLVE_GUIDELINES_MODE=consistency
+
+# Generate both
+export EVOLVE_GUIDELINES_MODE=both
+```
+
+Or for a one-off Phoenix sync, set `--guidelines-mode` to `regular`, `consistency`, or `both`:
+
+```bash
+uv run evolve sync phoenix --guidelines-mode consistency
+```
+
+### Configuring consistency guideline generation
+
+The consistency pipeline scores each decision step in a trajectory by resampling the decision multiple times and measuring how much the outcome varies, i.e. its uncertainty. The guideline-generation prompt is then steered toward the highest-uncertainty steps rather than summarizing the trajectory as a whole.
+
+Consistency guideline generation is noticeably more costly (multiple resample LLM calls per trajectory instead of one) and is worth it when you specifically want to catch agent behavior that's unstable across runs — decisions that the agent sometimes gets right and sometimes doesn't. 
+
+Tunable via:
+
+| Variable | Default | Description |
+|---|---|---|
+| `EVOLVE_HIGH_UNCERTAINTY_THRESHOLD` | `0.2` | Steps scoring above this are treated as high-uncertainty |
+| `EVOLVE_LOW_UNCERTAINTY_THRESHOLD` | `0.1` | Steps scoring below this are treated as stable |
+| `EVOLVE_SKIP_ON_NO_UNCERTAINTY` | `true` | Skip guideline generation entirely if no step exceeds the uncertainty threshold |
+
+The resampling behavior itself (sample count, per-step-type uncertainty metric) is defined in a YAML config file shipped alongside the consistency pipeline; advanced users calling `generate_consistency_guidelines()` directly from Python can point it at a custom config via `config_path=`.
+
+## Verifying output
+
+```bash
+uv run evolve entities list <namespace> --type guideline
+```
+
+Each guideline's `metadata.generation_method` is `"regular"` or `"consistency"`, so you can tell which pipeline produced it when running in `both` mode. See [Guideline Provenance](low-code-tracing.md#6-understanding-guideline-provenance-metadata) for the full metadata schema, including `creation_mode` (`auto-mcp` vs `auto-phoenix` vs `manual`).
+
+## See also
+
+- [Configuration](configuration.md) — model selection (`EVOLVE_GUIDELINES_MODEL`) and other environment variables
+- [Low-Code Tracing](low-code-tracing.md) — instrumenting your agent so traces reach Phoenix in the first place
+- [Phoenix Sync](phoenix-sync.md) — batch guideline generation from traced trajectories

docs/guides/low-code-tracing.md

@@ -205,6 +205,8 @@ uv run evolve sync phoenix \
     --include-errors
 ```
 
+See the [Phoenix Sync](phoenix-sync.md) guide for the full set of sync options, and [Enabling Guidelines](guidelines.md) for choosing between regular, consistency, or both guideline generation modes.
+
 ### 5. Verify Generated Guidelines
 
 ```bash

docs/guides/phoenix-sync.md

@@ -2,6 +2,8 @@
 
 Sync agent trajectories from Arize Phoenix to Evolve and automatically generate guidelines.
 
+This guide assumes traces are already reaching Phoenix — see [Low-Code Tracing](low-code-tracing.md) to instrument your agent first. For choosing *how* guidelines get generated (regular vs. consistency), see [Enabling Guidelines](guidelines.md).
+
 ## Overview
 
 The Phoenix sync module:
@@ -44,13 +46,17 @@ uv run evolve sync phoenix \
   --limit 500 \
   --include-errors
 
+# Generate consistency guidelines instead of regular ones
+uv run evolve sync phoenix --guidelines-mode consistency
+
 # Full options
 uv run evolve sync phoenix \
   --url http://localhost:6006 \
   --namespace production \
   --project my_project \
   --limit 200 \
-  --include-errors
+  --include-errors \
+  --guidelines-mode both
 ```
 
 ### CLI Options
@@ -62,6 +68,7 @@ uv run evolve sync phoenix \
 | `--project` | `-p` | Phoenix project name |
 | `--limit` | | Max spans to fetch (default: 100) |
 | `--include-errors` | | Include failed/error spans |
+| `--guidelines-mode` | | Guideline generation mode: `regular`, `consistency`, or `both` — see [Enabling Guidelines](guidelines.md) |
 
 ### Python API
 

docs/tutorials/guidelines-loop.md

@@ -0,0 +1,96 @@
+# Starter Example: Evolving Agent Loop
+
+This tutorial takes you through the end-to-end evolving agent loop: an existing agent runs and is traced, guidelines are generated from that trace, and those guidelines are then fed back into a *new* run of the same agent, so it actually learns from what it did before.
+
+This tutorial covers **full Evolve (MCP server / CLI)**. It doesn't apply to [Evolve Lite](../integrations/claude/evolve-lite.md), where the equivalent loop (`/evolve-lite:learn`, then automatic injection on the next prompt) is handled entirely by the host agent — see the [Claude Code starter tutorial](../examples/hello_world/claude.md) for that version.
+
+!!! note "Set expectations"
+    The example agent here (`examples/low_code/smolagents_demo.py`) does simple arithmetic with two tools, so there's little room for it to behave meaningfully differently after "learning" — don't expect a dramatic before/after. The goal of this tutorial is to demonstrate the full wiring (trace → sync → generate → retrieve → inject) end to end on a small, fast example. Point the same steps at a real agent with real failure modes to see guidelines actually change behavior.
+
+## What you'll build
+
+You'll run a small pipeline end to end, in five stages:
+
+1. **Run** — a smolagents agent (`smolagents_demo.py`) executes a task, with tracing enabled.
+2. **Trace** — that run is captured as a trajectory in Phoenix.
+3. **Generate** — `evolve sync phoenix` pulls the trajectory out of Phoenix and generates guideline entities from it.
+4. **Verify** — you'll confirm those guidelines were actually stored.
+5. **Retrieve & inject** — a second script (`guidelines_retrieval_demo.py`) fetches the relevant guidelines and re-runs the same agent with them injected as instructions, closing the loop.
+
+By the end, you'll have watched a guideline travel all the way from "something the agent did" to "something the agent is told to do differently next time."
+
+## Requirements
+
+- [`uv` installed](https://docs.astral.sh/uv/getting-started/installation/)
+- An LLM API key for your provider (e.g. `OPENAI_API_KEY`)
+
+Run every command below from the repo root (don't `cd` into `examples/low_code`, even to run the example scripts — Python adds a script's own directory to its import path automatically). This matters because the filesystem backend used throughout (`EVOLVE_BACKEND=filesystem`) stores data relative to the current directory; staying anchored at the repo root keeps every step reading and writing the same `evolve_data/`.
+
+## Step 0: Install dependencies
+
+```bash
+uv sync --extra examples
+```
+
+Arize Phoenix and `sentence-transformers` (used for guideline retrieval) are core dependencies and come with any `altk-evolve` install. This step additionally pulls in `smolagents`, the OpenInference tracing instrumentors, and the example scripts' other dependencies.
+
+## Step 1: Start Phoenix
+
+```bash
+uv run phoenix serve
+# Server runs at http://localhost:6006
+```
+
+## Step 2: Run the agent with tracing enabled
+
+```bash
+EVOLVE_AUTO_ENABLED=true \
+EVOLVE_TRACING_PROJECT=guidelines-tutorial \
+uv run python examples/low_code/smolagents_demo.py
+```
+
+`smolagents_demo.py` runs a `CodeAgent` with two tools (`add`, `multiply`) against the task "What is (5 * 5) + 10?". With `EVOLVE_AUTO_ENABLED=true`, [Low-Code Tracing](../guides/low-code-tracing.md) patches the agent's LLM calls so the full trajectory reaches Phoenix.
+
+Run it two or three times so there's more than one trajectory to generate guidelines from.
+
+## Step 3: Sync the trace into Evolve and generate guidelines
+
+```bash
+EVOLVE_BACKEND=filesystem \
+uv run evolve sync phoenix \
+    --project guidelines-tutorial \
+    --namespace guidelines-tutorial \
+    --guidelines-mode regular
+```
+
+See [Phoenix Sync](../guides/phoenix-sync.md) for the full set of sync options, and [Enabling Guidelines](../guides/guidelines.md) if you want to try `--guidelines-mode consistency` or `both` instead.
+
+## Step 4: Verify guidelines exist
+
+```bash
+EVOLVE_BACKEND=filesystem \
+uv run evolve entities list guidelines-tutorial --type guideline
+```
+
+You should see one or more `guideline` entities, each carrying `metadata.creation_mode: "auto-phoenix"` and `metadata.generation_method: "regular"`.
+
+## Step 5: Retrieve guidelines and re-run the agent with them injected
+
+```bash
+EVOLVE_BACKEND=filesystem \
+uv run python examples/low_code/guidelines_retrieval_demo.py --namespace guidelines-tutorial --task "What is (5 * 5) + 10?"
+```
+
+This script:
+
+1. Calls `EvolveClient().select_guidelines(namespace_id, task_query)` — the same dosage-aware core + top-k retrieval used by the `get_relevant_guidelines` MCP tool — to fetch guidelines relevant to the task.
+2. Formats the retrieved guideline content into a plain-text instructions block.
+3. Passes that block as `instructions=` when constructing a fresh `CodeAgent`, then re-runs the same task.
+
+The printed output shows exactly which guidelines were retrieved and injected before the agent runs, so you can confirm the loop closed even though the toy task's final answer won't visibly change.
+
+## What's next
+
+- Point `EVOLVE_TRACING_PROJECT` / `--project` at a real agent with real tool failures and retry loops — that's where injected guidelines start visibly changing behavior.
+- Swap `EVOLVE_BACKEND=filesystem` for Milvus or Postgres for persistent, semantic retrieval — see [Configuration](../guides/configuration.md).
+- Try `--guidelines-mode consistency` in Step 3 to generate guidelines focused on the trajectory's least-stable steps instead — see [Enabling Guidelines](../guides/guidelines.md).

docs/tutorials/index.md

@@ -29,3 +29,11 @@ Get started with Evolve on your platform of choice. Each tutorial walks through
 </div>
 
 [Full tutorial →](../examples/hello_world/codex.md)
+
+---
+
+## Starter Example: Evolving Agent Loop
+
+The starter examples above use Evolve Lite. This tutorial instead walks through **full Evolve (MCP server / CLI)**: taking an existing Python agent, tracing its runs, generating guidelines from those traces, and retrieving the guidelines back into a new run so the agent actually uses them.
+
+[Full tutorial →](guidelines-loop.md)

examples/low_code/guidelines_retrieval_demo.py

@@ -0,0 +1,85 @@
+import argparse
+import os
+
+from altk_evolve.config.llm import llm_settings
+from altk_evolve.frontend.client.evolve_client import EvolveClient
+
+from smolagents import CodeAgent, LiteLLMModel, tool
+
+# Reuse the same tools as smolagents_demo.py so this is a direct "part two" of that demo.
+from local_mcp_server import add as mcp_add, multiply as mcp_multiply
+
+
+@tool
+def add(a: int, b: int) -> int:
+    """
+    Add two numbers.
+    Args:
+        a: First number.
+        b: Second number.
+    """
+    return mcp_add(a, b)  # type: ignore[no-any-return,operator]
+
+
+@tool
+def multiply(a: int, b: int) -> int:
+    """
+    Multiply two numbers.
+    Args:
+        a: First number.
+        b: Second number.
+    """
+    return mcp_multiply(a, b)  # type: ignore[no-any-return,operator]
+
+
+def format_guidelines(selection) -> str:
+    """Turn retrieved guideline entities into a plain-text instructions block."""
+    lines = [f"- {g.content}" for g in selection.all]
+    if not lines:
+        return ""
+    return "Apply these lessons learned from previous runs:\n" + "\n".join(lines)
+
+
+def main():
+    parser = argparse.ArgumentParser(description="Re-run the smolagents demo with retrieved guidelines injected.")
+    parser.add_argument(
+        "--namespace",
+        default=os.environ.get("EVOLVE_NAMESPACE_ID", "evolve"),
+        help="Namespace guidelines were synced/saved into",
+    )
+    parser.add_argument("--task", default="What is (5 * 5) + 10?", help="Task to run the agent on")
+    args = parser.parse_args()
+
+    client = EvolveClient()
+    selection = client.select_guidelines(namespace_id=args.namespace, task_query=args.task)
+
+    print(f"Retrieved {len(selection.core)} core + {len(selection.retrieved)} task-specific guideline(s) from '{args.namespace}':")
+    for guideline in selection.all:
+        print(f"  - {guideline.content}")
+    if not selection.all:
+        print("  (none found - run the guideline-generation steps first)")
+
+    instructions = format_guidelines(selection)
+
+    # Same model configuration pattern as smolagents_demo.py
+    model_id = os.environ.get("EVOLVE_EXAMPLE_AGENT_MODEL") or llm_settings.guidelines_model
+    custom_provider = llm_settings.custom_llm_provider
+    model = LiteLLMModel(model_id=model_id, custom_llm_provider=custom_provider)
+
+    agent = CodeAgent(
+        tools=[add, multiply],
+        model=model,
+        add_base_tools=False,
+        instructions=instructions or None,
+    )
+
+    print("\nRunning Smolagents CodeAgent with guidelines injected as instructions...")
+    try:
+        result = agent.run(args.task)
+        print(f"Result: {result}")
+    except Exception as e:  # noqa: BLE001
+        raise SystemExit(f"Error running agent: {e}") from e
+
+
+if __name__ == "__main__":
+    main()

mkdocs.yaml

@@ -78,11 +78,13 @@ nav:
     - Starter Example (IBM Bob): examples/hello_world/bob.md
     - Starter Example (Claude Code): examples/hello_world/claude.md
     - Starter Example (Codex): examples/hello_world/codex.md
+    - Starter Example (Evolving Agent Loop): tutorials/guidelines-loop.md
   - Guides:
     - Configuration: guides/configuration.md
+    - Enabling Guidelines: guides/guidelines.md
+    - Low-Code Tracing: guides/low-code-tracing.md
     - Phoenix Sync: guides/phoenix-sync.md
     - Extract Trajectories: guides/extract-trajectories.md
-    - Low-Code Tracing: guides/low-code-tracing.md
     - Memory Hooks: guides/memory-hooks.md
     - Data Retention: guides/retention.md
   - Reference: