Building A Production-Ready Customer Service Chatbot

Or: How I Learnt LangChain, RAG And How To Build Custom MCP Servers Using FastMCP.

• Prototype: Prototype customer service chatbot could be integrated into a website. This integrates everything that was learnt during the experiments.
  • Goals:
    • Customers can identify themselves
    • Customers can ask general questions about the sales process
    • Customers can ask questions about their past and upcoming appointments
    • Customers can ask questions about their personal information
  • Features:
    • Chat loop with streaming output
    • Tool calling
    • RAG in Markdown and PDF files, multilingual PDF support
    • Custom MCP server to connect to internal web services
• Experiment 6: Test document retrieval (RAG) in multiple sources and in multiple formats (Markdown, PDF)
  • New technologies:
    • LangChain embedding model integrations
    • LangChain vector store integrations
    • LangChain text splitting (chunking)
    • paraphrase-multilingual-MiniLM-L12-v2 for embedding English, German and Dutch (previous all-MiniLM-L6-v2 only works for English)
    • ChromaDB to store embeddings
• Experiment 5: Prototype customer service chatbot that could be integrated into a website. Chat loop, tool calling, MCP, streaming output.
  • Goals:
    • Customers can identify themselves
    • Customers can ask general questions about the sales process
    • Customers can ask questions about their past and upcoming appointments
    • Customers can ask questions about their personal information
  • New technologies: FastMCP
  • Other technologies: HTTPX, Amazon Cognito
  • Parts of the MCP server code that call backend services were written using Claude Code
• Experiment 4: Chatbot that integrates everything learnt so far and adds MCP: agentic chatbot that provides recipes and time information. Chat loop, tool calling, RAG, MCP, streaming output.
  • New technologies: LangChain MCP adapters
• Experiment 3: Test for agentic chatbot that provides recipes using a RAG tool. No chat loop.
  • New technologies: LangChain agents and tools
• Experiment 2: Test streaming output tokens as they arrive. Sci-fi roleplay, no chat loop, no RAG.
  • New technologies: Token streaming
• Experiment 1: Initial chatbot that provides recipes based on my personal recipes using RAG. Chat loop where RAG context is hard inserted into every iteration. Context is not preserved between iterations.
  • New technologies: LangChain for the chatbot and HuggingFace integration; all-MiniLM-L6-v2 and LanceDB for the embedding model and store, respectively

The experiments were in part based on the following resources:

• https://dataengineeringcentral.substack.com/p/lance-for-embeddings-vector-database
• https://academy.langchain.com/courses/langchain-essentials-python