Learning Path Generator

An AI agent that generates a personalized, week-by-week learning path for any topic, then critiques and refines its own output before presenting it. Built with LangGraph to explore stateful, cyclical agent orchestration, as opposed to a single one-shot LLM call.

Around the core agent, the project adds the pieces a production AI system needs:

• RAG mode: plans can be grounded in the learner's own documents instead of the web
• Evaluation suite: fixed test requests with automatic checks, latency and token logging, and unit tests for the checks
• Human-in-the-loop: the graph can pause for a person to approve or reject a plan, and resume later from a saved checkpoint
• MCP server: the pipeline is exposed as tools any MCP client can call (tested from Claude Desktop)
• Docker Compose: the MCP server and n8n run together in containers, with volumes so saved state survives restarts
• n8n automation: a web form triggers the agent, and the plan is emailed to the user as formatted HTML
• Observability: LangSmith traces every run, node by node

A second implementation of the same pipeline, built in CrewAI, is included as a framework comparison. See Framework Comparison and COMPARISON.md.

What it does

Given a topic, a time budget in weeks, a level (beginner, intermediate, expert) and a goal (for example job hunting or general learning), the agent:

1. Searches the web for real, current learning resources, or retrieves from the user's own documents (RAG mode)
2. Plans a week-by-week sequence of those resources that fits the time budget and level
3. Generates a polished, human-readable write-up
4. Critiques its own output against the original constraints, and either:
   • approves it,
   • sends it back to replan (pacing or sequencing problems), or
   • sends it back to research (the resources themselves are weak)
5. Optionally pauses for human review, where a person approves the plan or rejects it with feedback
6. Finalizes the plan

The critique loop is capped at 3 rounds. If the cap is hit without approval, the plan is still returned, with a note that it may need manual review.

Architecture

START → (mode routing)
          ├─ "web"       → search   ─┐
          └─ "documents" → retrieve ─┴→ plan → generate → critique → (routing)
                                          ↑                            ├─ approve, review on  → human_review
                                          │                            ├─ approve, review off → finalize → END
                                          ├──────────── replan ────────┤
                                          │    research (back to search or retrieve, same mode)
                                          │                            └─ cap reached → finalize → END
                                          │
                                          └──── human rejects with feedback ── human_review ── approves → finalize → END

A StateGraph with 7 nodes: search, retrieve, plan, generate, critique, human_review and finalize.

• A conditional entry point routes to search (web) or retrieve (documents) based on mode.
• A conditional edge after critique routes on the verdict. research goes back to whichever resource node matches the original mode.
• human_review only runs when the caller sets human_review_enabled. Otherwise an approved plan goes straight to finalize, so the eval suite and the basic MCP tool never pause.

Two resource-gathering modes:

• web: live web search via Tavily (LangGraph) or Serper (CrewAI)
• documents: a local RAG pipeline. Documents are chunked (RecursiveCharacterTextSplitter), embedded with a free local Hugging Face model (all-MiniLM-L6-v2, no API key), stored in Chroma, and the most relevant chunks are retrieved by similarity search.

Both modes produce the same SearchResult structure, so plan, generate and critique don't know which mode produced their input. RAG mode was added with zero changes to the downstream loop.

Project structure

Learning-path-generator/
├── learning_path_core.py      # the pipeline: state, nodes, graph (single source of truth)
├── mcp_server.py              # MCP server exposing the pipeline as 3 tools
├── hitl_demo.py               # terminal demo of the human-in-the-loop flow
├── Dockerfile                 # container image for the MCP server
├── docker-compose.yml         # runs the MCP server and n8n together
├── requirements.txt           # everything, for local development
├── requirements-mcp.txt       # trimmed list for the Docker image
├── .env.example               # which API keys you need (no real values)
├── notebooks/
│   ├── Learning_path_generator.ipynb          # LangGraph walkthrough and test runs
│   └── Learning_path_generator_crewai.ipynb   # CrewAI version
├── eval/
│   ├── eval.py                # evaluation suite
│   ├── test_eval.py           # unit tests for the eval checks
│   └── results/               # saved eval results (baseline included)
├── n8n/
│   └── learning_path_workflow.json   # exported n8n workflow
├── docs/
│   └── architecture.png       # generated from the compiled graph
├── my_documents/              # sample files for documents (RAG) mode
├── COMPARISON.md              # LangGraph vs CrewAI write-up
└── README.md

Everything imports the pipeline from learning_path_core.py: the MCP server, the Docker image, the eval suite and the LangGraph notebook. The notebook used to contain its own copy of the code, which drifted out of date as features were added. It now imports the module, so a change is made in one place.

Setup

1. Clone the repo and install dependencies

git clone https://github.com/Parul671992/Learning-path-generator.git
cd Learning-path-generator
pip install -r requirements.txt

2. Get free API keys

Key	Used by	Get it at
GROQ_API_KEY	LLM calls (both versions, MCP server, eval)	console.groq.com (no credit card)
TAVILY_API_KEY	Web search (LangGraph version, MCP server)	tavily.com
SERPER_API_KEY	Web search (CrewAI version)	serper.dev
LANGSMITH_API_KEY	Optional tracing	smith.langchain.com

3. Create your .env file

Copy .env.example to .env in the project root and fill in your keys. .env is gitignored, so real keys are never committed.

4. Run the notebooks

• notebooks/Learning_path_generator.ipynb: LangGraph walkthrough. Sections 1 and 2 (setup and architecture diagram) make no LLM calls. Section 4 has test runs for web mode, step-by-step streaming, saved state and documents mode. Section 5 is a human-in-the-loop demo.
• notebooks/Learning_path_generator_crewai.ipynb: the CrewAI version.

Groq free tier note: the limit is 8,000 tokens per minute, and one run uses roughly 3,000 to 7,000 tokens. Wait about a minute between runs.

Evaluation

eval/eval.py runs fixed test requests through the real pipeline and checks each result automatically.

Test cases: 6 by default, covering beginner, intermediate and expert levels, 1-week to 12-week budgets, a niche topic, and one documents-mode case. A seventh, deliberately unrealistic case (ml_impossible: machine learning to LLMs in production in 1 week, for a beginner) exercises the replan loop and the iteration cap. It is marked expensive and only runs when named.

Checks per run:

Check	What it verifies
Plan validity	The plan matches the WeeklyPlan Pydantic schema and covers exactly weeks 1 to N
Grounding	Every resource in the plan comes from the search or retrieval results, so invented resources fail
Write-up links	Every link in the final write-up comes from the search results
Iterations	The critique loop stays within MAX_ITERATIONS, and runs that hit the cap without approval are flagged

Logged to CSV: latency, iteration count, input, output and total tokens (captured with LangChain's get_usage_metadata_callback, since structured output hides the raw token metadata), and pass or fail per check.

python eval/eval.py                  # all default cases
python eval/eval.py ml_impossible    # one named case
pytest eval/test_eval.py -v          # unit tests for the checks (no LLM calls, no tokens)

The script exits with code 1 if any case fails, so it can be used in CI. The 15 unit tests feed fake plans into each check to prove it catches what it should, for example an invented book, a missing week, an over-cap loop or an invented link.

Results: the baseline run (eval/results/eval_results_baseline_v1.csv) passed 6 of 6, at about 5 to 13 seconds and 3,500 to 7,200 tokens per case. The ml_impossible case behaved as designed: 3 iterations, cap reached without approval, about 15,000 tokens. Later runs on unchanged code sometimes failed grounding (search returned a Coursera search page, and the model turned it into specific course links), which shows why a single run per case isn't enough.

Human-in-the-loop

When human_review_enabled is set, the graph pauses after critique approves a plan, using LangGraph's interrupt(). Its full state is saved to a SQLite checkpoint file (checkpoints.db), keyed by thread_id.

• Approve: the plan goes to finalize.
• Reject with feedback: the plan goes back to plan. The feedback is stored in its own state field, human_feedback, which is never overwritten. Both plan and critique see it and treat it as top priority, and the critique round count resets so the new draft gets a full budget.
• Resume later: a completely new process can resume the review with just the thread_id, because the state lives in the checkpoint file, not in memory.

Try it with python hitl_demo.py, or Section 5 of the LangGraph notebook.

MCP server

mcp_server.py exposes the pipeline as 3 MCP tools:

Tool	What it does
generate_learning_path	One call, returns the finished plan (no human review)
start_learning_path_review	Runs until the plan pauses for review, returns the draft and a thread_id
resume_learning_path_review	Approves or rejects a paused draft by thread_id, returns the final plan or a revised draft

An MCP tool call is a single request and response, so it can't wait for a human. That's why review is split into a start tool and a resume tool, with the checkpoint file carrying state between them.

Local use with Claude Desktop (stdio)

1. Add an entry for this server to Claude Desktop's config (claude_desktop_config.json), with command pointing at your Python executable, args at the full path to mcp_server.py, and your API keys under env.
2. Restart Claude Desktop. The server should show as running under Settings, then Developer.
3. Ask Claude Desktop to generate a learning path, and it calls the tool.

With stdio, Claude Desktop starts mcp_server.py itself as a local process. The MCP_TRANSPORT environment variable switches the same file to network mode (streamable-http), which Docker uses. Claude Desktop never sets it, so the local setup is unaffected.

Docker Compose and n8n

docker-compose.yml runs two containers on a shared network:

• learning-path-mcp: the MCP server on port 8000, built from the Dockerfile
• n8n: the workflow automation tool, with its editor at http://localhost:5678

docker compose up --build -d     # build and start both
docker compose ps                # see what's running
docker compose logs -f           # follow the logs
docker compose down              # stop (volumes and saved data are kept)

Two named volumes keep data across restarts and rebuilds: checkpoints (paused reviews, via the CHECKPOINT_DB environment variable) and n8n_data (workflows and credentials). API keys come from .env at run time and are never baked into the image.

On the shared network, n8n reaches the MCP server by service name, at http://learning-path-mcp:8000/mcp.

The n8n workflow

Form trigger → MCP Client (generate_learning_path) → Markdown to HTML → Send Email

A person fills in a web form (topic, level, goal, weeks, email). n8n calls the agent through the MCP server, converts the Markdown plan to HTML, and emails it from a dedicated project Gmail account using an SMTP app password. Claude Desktop needs a person chatting and lets the AI decide whether to call the tool. n8n runs the same fixed steps every time a form is submitted, with no chat involved.

To use it, import n8n/learning_path_workflow.json into n8n and create your own SMTP credential. Credentials are not included in the export.

Docker image design choices

• The embeddings model is downloaded at build time, not at container startup, so the container never needs the network for it.
• PyTorch is pinned to the CPU-only build in requirements-mcp.txt, which avoids several gigabytes of unused CUDA packages.
• requirements-mcp.txt is a trimmed dependency list, so the image is smaller and builds faster.
• Secrets are passed at run time via env_file, never written into the image.

Observability (LangSmith)

With the three LANGSMITH_* lines from .env.example set, every run is traced automatically, with no code changes. LangSmith shows each node in order, critique's verdict and feedback, token use and timing per step. Traces appear for local runs, the eval suite and the Docker container (so runs triggered from n8n are visible too).

Design decisions and lessons learned

Pipeline design

• Temperature varies by node role. plan and critique use temperature 0 for consistent, evaluative work. generate uses 0.7 for natural, user-facing writing.
• Structured output where code consumes the result, free text where a person reads it. plan and critique use Pydantic schemas through .with_structured_output(). The final write-up is free text.
• Critique calibration was the hardest part. The first critique prompt said "be critical, not lenient" with no bar for good enough, so it never approved anything and hit the cap on every run. Explicit approval criteria fixed it, and runs now converge in 1 to 2 rounds. The general lesson: vague instructions to an LLM produce inconsistent behavior, while explicit, bounded ones produce reliable behavior. The same fix worked for generate's tone ("use emojis sparingly" became "do not use emojis").
• Retrieval being correct isn't enough. Even with retrieval grounded in the user's files, plan still invented familiar web courses until its prompt explicitly said to use only the provided resources.
• Adding a second path means rechecking every conditional edge. The research route originally always went to web search, so a documents-mode run could silently switch to the web mid-run.
• A retry wrapper (invoke_with_retry) lets occasional schema-validation failures self-heal. On rate-limit errors it waits 15 then 30 seconds before retrying, because retrying immediately just fails again.
• Search results replace rather than append on a research loop, a simplifying assumption for this version.

Evaluation

• The eval caught a real hallucination on its first runs. A plan listed a book that search never returned, despite the prompt forbidding it.
• A check can be wrong too. Documents-mode plans were flagged as ungrounded because the model wrote "file path: section heading" instead of the bare path. Matching on "starts with the source" fixed it, and empty URLs are filtered out so they can't make everything pass. Unit tests now cover both cases.
• Rate limits distort latency. Early eval runs got slower case by case because the client was waiting out Groq's per-minute limit, not because the pipeline was slower. A pause between cases made the numbers honest.

Human-in-the-loop

• The AI reviewer overrode the human at first. After a rejection ("use only YouTube videos"), critique never saw the human's feedback and kept rejecting compliant plans, and its own feedback overwrote the human's each round. The fix: a separate human_feedback field that both plan and critique must respect. In a HITL system, the human's instruction has to outrank the AI reviewer.
• A rejection used to count against the critique cap, so a revised draft could hit the cap and skip the human entirely. A rejection now resets the round count.
• In-memory checkpoints can't support a pause. The checkpointer moved from :memory: SQLite to a file, so a paused review survives the process ending.
• Checkpoint loading is allowlisted. LangGraph warns that rebuilding arbitrary classes from a checkpoint file is a security risk and will be blocked in future versions. The serializer now explicitly allows only SearchResult and PlanItem.

Infrastructure

• Extracting the notebook into a module paid off repeatedly. It was first needed for MCP, then for Docker, the eval suite and HITL. The notebook now imports it too, so code no longer drifts between copies.
• A local dependency making a hidden network call caused intermittent failures. The Hugging Face library checked the Hub on every start, which sometimes blew past Claude Desktop's connection timeout. Fixed with HF_HUB_OFFLINE=1, which must be set before any Hugging Face import, and in Docker by downloading the model at build time.

For CrewAI-specific lessons, including five framework bugs found and fixed, see COMPARISON.md.

Known limitations and next steps

• Runs only on my laptop. Deploying to a cloud host (for example AWS) would make the form public, which would also need a login or rate limit and HTTPS.
• No content-quality scoring. The eval checks structure and grounding, not whether a plan is good. An LLM-as-judge with a rubric, using a different model from the critique node, would cover this.
• One run per test case. Results vary between runs, so reporting a pass rate over several runs per case would be more reliable.
• Users can't upload documents through the form. Documents mode works, but only with files already in my_documents. A shared volume between n8n and the MCP container would enable uploads.
• Name-only resources aren't checked in the write-up. The link check catches invented URLs, but not an invented resource mentioned by name only.
• Single LLM provider (Groq, free tier), with no fallback.
• Search result count is fixed at 5, kept the same as the CrewAI version for a fair comparison.
• resource_type stays "unknown" for web results, since classifying it would cost an extra LLM call per search.
• The CrewAI version is not exposed as an MCP server.

Tech stack

• LangGraph: stateful agent orchestration, interrupts and checkpointing
• CrewAI: role-based agent orchestration (comparison version)
• Groq (openai/gpt-oss-120b): LLM inference
• Tavily (langchain-tavily): web search for the LangGraph version and MCP server
• Serper: web search for the CrewAI version
• Chroma and Hugging Face all-MiniLM-L6-v2: local vector store and embeddings for RAG
• Pydantic: structured output schemas
• SQLite (SqliteSaver): persistent checkpoints for human-in-the-loop
• MCP (mcp[cli], FastMCP): exposes the pipeline as tools for any MCP client
• Docker and Docker Compose: containerized MCP server and n8n
• n8n: workflow automation (form to agent to email)
• LangSmith: tracing and observability
• pytest: unit tests for the eval checks

Framework Comparison: LangGraph vs CrewAI

Both versions are in notebooks/: Learning_path_generator.ipynb (LangGraph) and Learning_path_generator_crewai.ipynb (CrewAI). Summary below, full details in COMPARISON.md.

• Looping and routing: LangGraph has native conditional edges for cycles. CrewAI's sequential mode has no equivalent, so the critique loop needed a manual outer Python loop. CrewAI's hierarchical mode looked like a fit but hit a documented framework bug where the manager agent does the work itself instead of delegating.
• Control-flow guarantees: in both frameworks, reliable stopping and routing logic has to live in code, not in an LLM's judgment.
• Bugs found: the CrewAI build surfaced 5 real bugs, including a Groq compatibility issue, a loose tool schema, the hierarchical delegation bug, a broken structured-output mechanism, and a stale task-caching bug that silently broke the critique loop.
• Persistence: LangGraph has built-in checkpointing, which made human-in-the-loop possible. CrewAI has no equivalent, so a failed run restarts from the first task.