blog: iroh for distributed LLM inference (Mesh LLM)

.gitignore

@@ -27,8 +27,12 @@ yarn-debug.log*
 yarn-error.log*
 
 # local env files
+.env
 .env*.local
 
+# playwright MCP debug output
+.playwright-mcp/
+
 # vercel
 .vercel
 

next.config.mjs

@@ -17,6 +17,7 @@ const redirects = async () => {
   return [
     // Solutions page renames
     { source: '/solutions/rave', destination: '/solutions/video', permanent: true },
+    { source: '/solutions/nous', destination: '/solutions/distributed-ai', permanent: true },
 
     // Enterprise & support redirects
     { source: '/enterprise', destination: '/services/enterprise', permanent: true },

src/app/blog/mesh-llm/page.mdx

@@ -0,0 +1,104 @@
+import { BlogPostLayout } from '@/components/BlogPostLayout'
+
+export const post = {
+  draft: false,
+  author: 'n0 team',
+  date: '2026-07-11',
+  title: 'Mesh LLM: distributed AI computing on iroh',
+  description:
+    'How Mesh LLM pools existing GPU resources across machines into a single OpenAI-compatible API, built on iroh.',
+}
+
+export const metadata = {
+    title: post.title,
+    description: post.description,
+    openGraph: {
+      title: post.title,
+      description: post.description,
+      images: [{
+        url: `/api/og?title=Blog&subtitle=${post.title}`,
+        width: 1200,
+        height: 630,
+        alt: post.title,
+        type: 'image/png',
+      }],
+      type: 'article'
+    }
+}
+
+export default (props) => <BlogPostLayout article={post} {...props} />
+
+<img src="/blog/mesh-llm/header.png" alt="Distributed Model Inference: a mesh of laptop, GPU rig, mini PC, server, workstation, and cloud nodes connected directly to each other" className="w-full rounded-lg not-prose mb-8" />
+
+When people picture running a large language model, they picture a data center. Racks of GPUs that belong to someone else, a metered API, and a bill that grows every month you succeed. You send your prompts off to a black box and hope the price, the model, and the privacy policy all stay the way they were when you signed up.
+
+For a lot of teams that is a bad trade. You give up control over when models change, where your data goes, and what hardware runs your workloads. And as usage grows, so does the bill, with no lever to pull except "pay more."
+
+[Mesh LLM](https://meshllm.cloud) is a different shape. It pools the GPUs and memory you already have, across as many machines as you want to add, and exposes the whole thing as one OpenAI-compatible API. Start one node. Add more later. Let the mesh decide whether a model runs on the box in front of you, routes to a peer, or splits across several machines.
+
+## The problem: AI is expensive, and it is somebody else's
+
+The popular models are monoliths. Most people reach them through a UI or an API key and pay a large provider to run everything. That is convenient, and it is also a surrender. You do not control when the model gets updated, what memory it runs in, or what hardware sits underneath.
+
+Plenty of businesses and services that depend on these models want the opposite: more control, more pluggability, lower cost. They have GPUs sitting in offices, in closets, under desks. What they are missing is a way to make those machines act like one.
+
+## Mesh LLM: run the models yourself
+
+The pitch is simple. Run bigger models without buying bigger GPUs. Share compute privately with your team, or publicly with the world, to power agents and chat. Point any OpenAI client at http://localhost:9337/v1 and stop caring where the work actually happens.
+
+Under the hood, Mesh LLM distributes model compute across a mesh of iroh endpoints. A request can be served three ways:
+- Run it locally, on this machine's GPU.
+- Route it to a peer that already has the model loaded.
+- Split a model too big for any single box across several machines, as a pipeline.
+
+## How it works
+
+The architecture is pluggable. Plugins declare what they provide in a manifest, the runtime starts them, routes calls, and exposes their capabilities over MCP, HTTP, inference, and mesh events. The catalog ships with 40+ models, from half-a-billion-parameter models that fit on a laptop to 235B mixture-of-experts giants.
+
+For the giants, Mesh LLM has a split mode (internally, "Skippy"). A model gets partitioned by layer ranges into stages: layers 0 to 15 on one node, 16 to 31 on the next, and so on down the pipeline. Activations flow from one stage to the next, so several modest machines can run a model none of them could hold alone. The OpenAI client never sees any of this. It still just talks to localhost.
+
+## How it uses iroh
+
+Every node, whether it serves models or only sends requests, boots an iroh endpoint. That endpoint is the node's identity, a public key, and its only network surface. There is no central server. iroh handles the hole-punching, NAT traversal, and relay fallback needed to open a direct, authenticated QUIC connection between any two nodes, wherever they sit.
+
+To keep that working across the open internet, Mesh LLM runs two iroh relays in different regions, so nodes that cannot reach each other directly always have a fallback path nearby.
+
+The whole protocol rides on QUIC's ALPN negotiation. There are three:
+| ALPN | What it carries |
+|------|----------------|
+| mesh-llm/1 | Main mesh: gossip, routing, HTTP tunnels, plugin channels |
+| mesh-llm-control/1 | Owner control plane (config sync, ownership attestation) |
+| skippy-stage/2 | Latency-sensitive activation transport for split models |
+
+
+Inside the main `mesh-llm/1` connection, everything is a bidirectional QUIC stream tagged with a single leading byte that says what kind of stream it is. One connection carries gossip, inference, route queries, and peer-lifecycle events, all demuxed by that first byte:
+
+| Byte | Stream type | Description |
+|------|-------------|------------|
+| 0x01 | GOSSIP               | peer announcements (models, GPU, RTT, capabilities) |
+| 0x04 | TUNNEL_HTTP          | inference requests proxied to a peer |
+| 0x05 | ROUTE_REQUEST        | "which models do you host?" |
+| 0x06 | PEER_DOWN            | dead-peer notification |
+| 0x07 | PEER_LEAVING         | graceful shutdown |
+| 0x08 | PLUGIN_CHANNEL       | plugin RPC |
+| 0x0e | DIRECT_PATH_REQUEST  | share direct addresses for NAT traversal |
+
+
+The neat part is what this buys you. iroh gives authenticated, NAT-traversing QUIC between any two machines, addressed by public key. So "route to a peer" and "stream activations to the next pipeline stage" become the same primitive as "talk to localhost," just with a different endpoint ID. The networking stops being something you have to think about.
+
+iroh provides the secure transport. Mesh LLM builds its own gossip layer on top, so it controls exactly who gets admitted to the mesh, which versions are compatible, and which peers to trust.
+
+
+## Getting started
+
+Users can install the lightweight software (about 18 MB) and either join the
+public mesh or configure private deployments. The system presents itself as
+`localhost:9337/v1` to any standard OpenAI client.
+
+A mobile app is coming, built on iroh's Swift SDK. The plan is to speak ACP, the
+emerging agent standard, so other clients can join the mesh too. The throughline
+is the same one that motivated the whole project: more peer to peer, fewer
+closed servers, and no lock-in.  
+
+- [See the code](https://github.com/Mesh-LLM/mesh-llm)
+- [Mesh LLM Website](https://meshllm.cloud) 

src/app/page.jsx

@@ -123,7 +123,7 @@ export default function Page() {
                       <circle r="1" cx="1" cy="1" />
                     </svg>
                     <div className="text-gray-400">Principal Engineer at Nous</div>
-                    <a href="/solutions/nous" className="text-irohPurple-500 hover:underline ml-2">Read the Case Study <ArrowRightIcon className='inline-block w-5 h-5 ml-1 -mt-1' /></a>
+                    <a href="/solutions/distributed-ai" className="text-irohPurple-500 hover:underline ml-2">Read the Case Study <ArrowRightIcon className='inline-block w-5 h-5 ml-1 -mt-1' /></a>
                   </div>
                 </figcaption>
               </figure>
@@ -177,7 +177,7 @@ export default function Page() {
               <h3 className='text-5xl font-bold mb-2'>How are people using iroh?</h3>
             </div>
             <div className='grid gap-6 md:grid-cols-2'>
-              <Link href="/solutions/nous" className="block h-full">
+              <Link href="/solutions/distributed-ai" className="block h-full">
                 <div className="h-full p-6 border border-irohGray-300 dark:border-irohGray-700 rounded-lg hover:border-irohPurple-500 transition-colors">
                   <p className="text-xl font-medium text-irohGray-800 dark:text-irohGray-100 mb-2">Distributed AI Training</p>
                   <p className="text-irohGray-600 dark:text-irohGray-300">Train foundation LLMs with compute distributed around the world, across AWS, GCP, Azure, and self-hosted infrastructure.</p>

src/app/services/enterprise/page.tsx

@@ -221,10 +221,10 @@ export default function EnterprisePage() {
                 <p className="text-sm text-irohGray-500 dark:text-irohGray-400">Real-time video for millions of concurrent connections.</p>
                 <p className="text-sm text-irohPurple-500 mt-3">Read the case study &rarr;</p>
               </Link>
-              <Link href="/solutions/nous" className="block p-6 rounded-lg border border-irohGray-300 dark:border-irohGray-700 hover:border-irohPurple-500 transition-colors">
+              <Link href="/solutions/distributed-ai" className="block p-6 rounded-lg border border-irohGray-300 dark:border-irohGray-700 hover:border-irohPurple-500 transition-colors">
                 <img src="/img/user-logos/nous.png" alt="Nous logo" className="object-contain max-h-10 mb-4" />
-                <p className="font-semibold mb-1">Distributed AI Training</p>
-                <p className="text-sm text-irohGray-500 dark:text-irohGray-400">Train foundation LLMs across AWS, GCP, Azure, and self-hosted compute.</p>
+                <p className="font-semibold mb-1">Distributed AI Training &amp; Inference</p>
+                <p className="text-sm text-irohGray-500 dark:text-irohGray-400">Train foundation LLMs across AWS, GCP, Azure, and self-hosted compute, or pool GPUs into a single inference API.</p>
                 <p className="text-sm text-irohPurple-500 mt-3">Read the case study &rarr;</p>
               </Link>
               <Link href="/solutions/pos" className="block p-6 rounded-lg border border-irohGray-300 dark:border-irohGray-700 hover:border-irohPurple-500 transition-colors">

src/app/solutions/nous/page.jsx → src/app/solutions/distributed-ai/page.jsx

@@ -3,35 +3,33 @@ import { HeaderSparse } from '@/components/HeaderSparse'
 import { FooterMarketing } from "@/components/FooterMarketing"
 import { OpenSourceIllustration } from "@/components/OpenSourceIllustration"
 import Link from "next/link"
-import { Server, Zap, Globe, Shield, Cpu, MessageSquare, BarChart3, Network } from "lucide-react"
+import { Server, Zap, Globe, Shield, Cpu, Network } from "lucide-react"
 
 export const metadata = {
-  title: 'Nous Research - Use Case | Iroh',
-  description: 'How Nous uses iroh to train foundation LLMs with compute distributed around the world.',
+  title: 'Distributed AI - Use Case | Iroh',
+  description: 'How Nous and Mesh LLM use iroh to train foundation LLMs and pool GPUs across machines, with compute distributed around the world instead of centralized in one data center.',
 }
 
-export default function NousUseCasePage() {
+export default function DistributedAIUseCasePage() {
   return (
     <div>
       <HeaderSparse />
 
       <div className="min-h-screen transition-colors font-space bg-irohGray-50 dark:bg-black text-irohGray-700 dark:text-irohGray-100">
         {/* Hero Section */}
         <section className="py-24 px-6 border-b border-irohGray-300 dark:border-irohGray-800 relative overflow-hidden">
-          {/* Background Logo */}
-          <div className="absolute right-0 top-1/2 -translate-y-1/2 w-[500px] h-[500px] opacity-10 pointer-events-none">
-            <img src="/img/user-logos/nous.png" alt="" className="w-full h-full object-contain" />
-          </div>
           <div className="container mx-auto max-w-6xl pt-12 relative z-10">
             <div className="grid lg:grid-cols-2 gap-16 items-center">
               <div>
                 <p className="text-irohPurple-500 font-medium mb-4 uppercase tracking-wide">Use Case: AI/ML</p>
                 <h1 className="text-5xl md:text-6xl mb-6 leading-tight font-bold">
-                  Distributed AI Training at Global Scale
+                  Distributed AI, Training to Inference
                 </h1>
                 <p className="text-xl text-irohGray-600 dark:text-irohGray-300 mb-8 leading-relaxed">
-                  Nous Research uses iroh to train foundation LLMs with compute distributed around the world,
-                  across AWS, GCP, Azure, and self-hosted infrastructure.
+                  <a href="https://nousresearch.com" target="_blank" rel="noopener noreferrer" className="text-irohPurple-500 hover:underline">Nous Research</a> uses
+                  iroh to train foundation LLMs with compute distributed around the world, across AWS, GCP,
+                  Azure, and self-hosted infrastructure. <a href="https://meshllm.cloud" target="_blank" rel="noopener noreferrer" className="text-irohPurple-500 hover:underline">Mesh LLM</a> uses
+                  it to pool GPUs across machines into a single inference API. Neither needed a data center to do it.
                 </p>
                 <div className="flex gap-4 flex-wrap">
                   <Link href="https://docs.iroh.computer/quickstart">
@@ -44,9 +42,9 @@ export default function NousUseCasePage() {
                       Visit Nous Research
                     </Button>
                   </Link>
-                  <Link href="https://docs.psyche.network" target="_blank" rel="noopener noreferrer">
+                  <Link href="https://meshllm.cloud" target="_blank" rel="noopener noreferrer">
                     <Button arrow="none" variant="outline" className="border-irohGray-300 dark:border-irohGray-600 px-6 py-2 text-sm font-medium cursor-pointer bg-transparent">
-                      Psyche Docs
+                      Visit Mesh LLM
                     </Button>
                   </Link>
                 </div>
@@ -58,9 +56,10 @@ export default function NousUseCasePage() {
           </div>
         </section>
 
-        {/* Results */}
+        {/* Nous: Results */}
         <section className="py-16 px-6 border-b border-irohGray-300 dark:border-irohGray-800">
           <div className="container mx-auto max-w-5xl">
+            <p className="text-irohPurple-500 font-medium mb-8 uppercase tracking-wide text-sm text-center">Nous Research: Training</p>
             <div className="grid md:grid-cols-4 gap-8 text-center">
               <div>
                 <p className="text-5xl font-bold text-irohPurple-500 mb-2">10x</p>
@@ -171,19 +170,75 @@ export default function NousUseCasePage() {
           </div>
         </section>
 
+        {/* Mesh LLM: Inference */}
+        <section className="py-20 px-6 border-t border-irohGray-300 dark:border-irohGray-800">
+          <div className="container mx-auto max-w-6xl">
+            <p className="text-irohPurple-500 font-medium mb-3 uppercase tracking-wide text-sm">Mesh LLM: Inference</p>
+            <h2 className="text-3xl font-bold mb-6">Pool GPUs Into One API</h2>
+            <div className="grid lg:grid-cols-2 gap-16 items-start">
+              <div>
+                <p className="text-lg text-irohGray-600 dark:text-irohGray-300 mb-6 leading-relaxed">
+                  <a href="https://meshllm.cloud" className="text-irohPurple-500 hover:underline">Mesh LLM</a> pools
+                  existing GPU resources across multiple machines and exposes them through a single
+                  OpenAI-compatible API, with no central server coordinating the mesh. Every node runs an
+                  iroh endpoint, so nodes connect directly to each other over authenticated QUIC instead of
+                  routing through a coordinator.
+                </p>
+                <div className="bg-irohGray-100 dark:bg-irohGray-800 p-6 rounded-lg">
+                  <p className="text-xl font-medium text-irohGray-800 dark:text-irohGray-100 italic">
+                    &ldquo;Control over when models change, where your data goes, and what hardware runs
+                    your workloads.&rdquo;
+                  </p>
+                </div>
+              </div>
+              <div className="space-y-6">
+                <div className="flex items-start gap-4">
+                  <Server className="h-8 w-8 text-irohPurple-500 shrink-0 mt-1" />
+                  <div>
+                    <h3 className="text-lg font-medium mb-2">Local Execution</h3>
+                    <p className="text-irohGray-600 dark:text-irohGray-300">
+                      A request runs on the requesting machine&apos;s GPU, if it&apos;s big enough.
+                    </p>
+                  </div>
+                </div>
+                <div className="flex items-start gap-4">
+                  <Network className="h-8 w-8 text-irohPurple-500 shrink-0 mt-1" />
+                  <div>
+                    <h3 className="text-lg font-medium mb-2">Peer Routing</h3>
+                    <p className="text-irohGray-600 dark:text-irohGray-300">
+                      Otherwise, it routes directly to a peer node that already has the model loaded.
+                    </p>
+                  </div>
+                </div>
+                <div className="flex items-start gap-4">
+                  <Shield className="h-8 w-8 text-irohPurple-500 shrink-0 mt-1" />
+                  <div>
+                    <h3 className="text-lg font-medium mb-2">Pipeline Splitting</h3>
+                    <p className="text-irohGray-600 dark:text-irohGray-300">
+                      Models too large for any single machine get partitioned by layer ranges across
+                      multiple nodes, so modest machines can collectively run what none of them could load alone.
+                    </p>
+                  </div>
+                </div>
+              </div>
+            </div>
+          </div>
+        </section>
+
         {/* Why Relays Matter */}
         <section className="py-20 px-6 border-t border-irohGray-300 dark:border-irohGray-800">
           <div className="container mx-auto max-w-4xl">
             <h2 className="text-3xl font-bold mb-6">Why Relays Matter</h2>
             <p className="text-lg text-irohGray-600 dark:text-irohGray-300 mb-6 leading-relaxed">
               Nous runs 5 iroh relays to ensure reliable connectivity across their distributed training network.
-              The key insight: <strong>when things deteriorate, they can&apos;t break</strong>.
+              Mesh LLM runs two, in different regions, for the same reason. The key insight: <strong>when things
+              deteriorate, they can&apos;t break</strong>.
             </p>
             <p className="text-lg text-irohGray-600 dark:text-irohGray-300 mb-6 leading-relaxed">
               Iroh automatically establishes direct connections when possible for maximum throughput. When direct
               connections aren&apos;t possible&mdash;due to NATs, firewalls, or network conditions&mdash;traffic
-              flows through relays. This fallback mechanism means training runs continue even when network
-              conditions change.
+              flows through relays. This fallback mechanism means training and inference keep running even when
+              network conditions change.
             </p>
           </div>
         </section>
@@ -228,7 +283,7 @@ export default function NousUseCasePage() {
               Get started with iroh in minutes. No complex configuration required.
             </p>
             <div className="flex flex-col sm:flex-row gap-4 justify-center">
-              <Link href="https://services.iroh.computer/signup?utm_source=website&utm_content=nous-cta">
+              <Link href="https://services.iroh.computer/signup?utm_source=website&utm_content=distributed-ai-cta">
                 <Button
                   arrow="none"
                   className="bg-irohGray-800 hover:bg-irohGray-700 text-irohPurple-500 px-8 py-3"