Rate-limit and abuse-protect public MCP reads and LLM endpoints

[harrymove-ctrl]

Context

The only throttle today is the 1/day demo-IP quota on /api/demo/extractions (consumeDemoQuota, apps/api/src/worker.ts:4296). Token-less public namespace MCP reads (/mcp, worker.ts:800) and the LLM-backed endpoints — /api/memwal/chat (worker.ts:634/3498), aiQueryRun (worker.ts:1349), /api/memwal/recall (worker.ts:631) — are completely uncapped: an open cost/abuse vector that blocks flipping the repo public (docs/launch-checklist.md).

Goal / user story

As the platform owner, I want per-IP and per-token rate limits on public reads and LLM endpoints so a single client can't run up unbounded Workers AI / OpenRouter cost or scrape the directory, returning clear 429s.

Acceptance criteria

• A rateLimit(env, key, { limit, windowSec }) helper returns { allowed, remaining, resetAt }, implemented via Cloudflare's Rate Limiting binding (preferred) or a D1 sliding window mirroring consumeDemoQuota.
• Distinct buckets: anonymous (per-IP via clientIp, worker.ts:4313) vs authed (per read-token/account) with higher authed limits.
• Enforced on: public MCP reads (/mcp), /api/memwal/chat, aiQueryRun, /api/memwal/recall, and the directory listing (/api/directory, worker.ts:687).
• On limit: HTTP 429 with Retry-After and a typed code (reuse the statusError(..., 429, "RATE_LIMITED") shape, see worker.ts:4302); X-RateLimit-Remaining/-Reset headers on success.
• Limits are env-configurable via wrangler.jsonc vars (e.g. RATE_LIMIT_CHAT_PER_MIN) with safe defaults; staging can override.
• Tests cover: under-limit passes, over-limit 429s, authed bucket > anon bucket, and limits are per-key not global.

Implementation notes

The Cloudflare Rate Limiting binding needs a ratelimit/[[unsafe.bindings]] entry in wrangler.jsonc (both top-level and env.staging); document the binding in .env.example/README. If the binding is awkward under tests, fall back to a D1 token-bucket keyed ${windowStart}:${ipHashOrToken}:${endpoint} reusing the upsert at worker.ts:4304. Keep this separate from the usage ledger (rate-limit = ephemeral sliding window; ledger = durable audit), though you may emit a rate_limited ledger event for abuse visibility. Do not regress the existing SSRF guard (isPrivateIpv4, worker.ts:4288).

Sui Overflow angle

A public hackathon demo with an open, unauthenticated MCP + LLM surface is a guaranteed cost/abuse incident the moment the link is shared. Capping it is what makes it safe to flip the repo public and submit to the Smithery/Claude/Cursor marketplaces (docs/launch-checklist.md D5) — it unblocks the growth/demo loop.

Dependencies

Per-token/per-account buckets get stronger once the accounts/owner-auth issue lands, but per-IP limiting ships independently. None blocking.

Part of the ContextMEM roadmap (#4) • Sui Overflow build.

[harrymove-ctrl]

Implementation Plan

Approach
Add a binding-agnostic rateLimit(env, key, { limit, windowSec }) helper backed by a D1 fixed-window counter that mirrors consumeDemoQuota (apps/api/src/worker.ts:4296) and returns { allowed, remaining, resetAt }. Enforce it from a single chokepoint — the router routeWorkerRequest (worker.ts:575) — wrapping each protected route so the handler bodies stay untouched. Use two bucket classes: anon keyed by hashed clientIp (worker.ts:4313) and authed keyed by the validated read-token hash / MemWal ownerId, each with its own env-var limit (authed > anon). Key design decision: go D1-first rather than the native Cloudflare Rate Limiting binding — the native binding only returns { success } (no remaining/resetAt), bakes its limit into config (can't be driven by a RATE_LIMIT_*_PER_MIN var), and isn't exercisable in vitest/Miniflare — all three are hard acceptance criteria. Keep the helper signature binding-shaped so a CF ratelimit binding can be slotted in later for the hottest unauth path.

Prerequisites & dependencies

• No blocking issues. Per-IP limiting ships independently (issue states "None blocking"). Per-token/account buckets get stronger once real accounts land, but the MemWal-delegate ownerId (readHostedRunAuth, worker.ts:1461) and ctxm_ read tokens already give us a usable authed key today.
• No new bindings required for the D1 path (reuses CONTEXTMEM_DB). If the team later wants the CF binding accelerator, it needs an unsafe.bindings entry of type: "ratelimit" in both top-level and env.staging of cloudflare/wrangler.jsonc — document but do not depend on it.
• New env vars (all optional, safe defaults) added to WorkerEnv (worker.ts:32-55) and cloudflare/wrangler.jsonc vars (top-level + env.staging).
• Deploy reminder (per repo memory): Worker is not shipped by CI — after merge run wrangler deploy --config apps/api/cloudflare/wrangler.jsonc and --env staging, and apply migration 0007 to both D1 dbs.

Step-by-step

1. DB migration. Create apps/api/migrations/0007_rate_limits.sql (next number after 0006):

   CREATE TABLE IF NOT EXISTS contextmem_rate_limits (
     bucket_key TEXT PRIMARY KEY,        -- `${windowSec}:${windowStart}:${endpoint}:${idHash}`
     endpoint TEXT NOT NULL,
     window_start INTEGER NOT NULL,      -- epoch seconds, floored to window
     count INTEGER NOT NULL DEFAULT 0,
     updated_at TEXT NOT NULL
   );
   CREATE INDEX IF NOT EXISTS idx_rate_limits_window ON contextmem_rate_limits(window_start);

   Mirror contextmem_demo_limits shape (migrations/0004_growth_loops.sql:12).

2. Core helper. In worker.ts near consumeDemoQuota (~4296) add:

   • async function rateLimit(env, key, { limit, windowSec }): Promise<{ allowed; remaining; resetAt }> — compute now = Math.floor(Date.now()/1000), windowStart = now - (now % windowSec), resetAt = (windowStart + windowSec) * 1000, bucketKey = ${windowSec}:${windowStart}:${key}. Atomic increment-and-read via the upsert pattern at worker.ts:4304 plus RETURNING count:

     INSERT INTO contextmem_rate_limits (bucket_key, endpoint, window_start, count, updated_at)
     VALUES (?, ?, ?, 1, ?)
     ON CONFLICT(bucket_key) DO UPDATE SET count = count + 1, updated_at = excluded.updated_at
     RETURNING count

     allowed = count <= limit; remaining = Math.max(0, limit - count). Counting rejected hits too is intentional (keeps hammered keys blocked).
   • function rateLimitConfig(env) — parse RATE_LIMIT_* vars to numbers with defaults; honor RATE_LIMIT_ENABLED !== "false".
   • async function rateLimitKey(request, endpoint, opts) — resolve identity: if readHostedRunAuth(request) (worker.ts:1461) present → acct:${ownerId} (authed); else if a validated read token is known → tok:${sha256Hex(token)} (authed); else ip:${await sha256Hex(clientIp(request))} (anon). Return { key: ${endpoint}:${idPart}, authed }. Reuse sha256Hex (worker.ts:4507).
   • async function enforceRateLimit(request, env, endpoint, { authedLimit, anonLimit, windowSec }) → returns the rateLimit result + authed flag.
   • function rateLimited429(rl, isMcp) — for MCP return mcpJsonError(message, 429) (worker.ts:5281); else build a Response directly (do not throw statusError, so we control headers) with body shaped like statusError(..., 429, "RATE_LIMITED") → { error: { code: "RATE_LIMITED", message, hint }, message }, status 429, headers Retry-After: ceil((resetAt-now)/1000), X-RateLimit-Remaining: 0, X-RateLimit-Reset: resetAt, wrapped in cors().
   • function withRateLimitHeaders(res, rl) — clone response, set X-RateLimit-Remaining/X-RateLimit-Reset.

3. Wire enforcement into the router (routeWorkerRequest, worker.ts:575), each as a one-liner guard before the existing handler call:

   • /api/memwal/recall (worker.ts:631), /api/memwal/chat (worker.ts:634): const rl = await enforceRateLimit(request, env, "chat"|"recall", ...); if (!rl.allowed) return rateLimited429(rl, false); return withRateLimitHeaders(await memwalChat(request, env), rl);
   • aiQueryRun in the hostedRunRoute block (worker.ts:652): same wrapper, endpoint "ai_query". (Demo runs key by anon IP since ownerId is demo:<ip>; authed runs key by ownerId.)
   • /api/directory (worker.ts:687): wrap listDirectoryNamespaces (anon-heavy, the scrape target), endpoint "directory".
   • Public MCP reads, MCP branch (worker.ts:772-795): enforce after authorizeNamespace succeeds (so a valid read token yields the authed bucket) but before createMcpHandler. Skip the GET browser-landing path (worker.ts:774-776). On block return rateLimited429(rl, true). Endpoint "mcp".
   • Leave createDemoExtraction/consumeDemoQuota as-is (separate durable quota; do not double-enforce).

4. Expose headers to browsers. In cors() (worker.ts:5303) extend access-control-expose-headers from "mcp-session-id" to also include x-ratelimit-remaining, x-ratelimit-reset, retry-after, else fetch clients can't read them.

5. Env + config. Add to WorkerEnv (worker.ts:32): RATE_LIMIT_ENABLED?, RATE_LIMIT_WINDOW_SEC?, RATE_LIMIT_MCP_ANON_PER_MIN?/_AUTHED_, RATE_LIMIT_CHAT_PER_MIN?/_AUTHED_, RATE_LIMIT_RECALL_PER_MIN?/_AUTHED_, RATE_LIMIT_AI_QUERY_PER_MIN?/_AUTHED_, RATE_LIMIT_DIRECTORY_PER_MIN? (all string since wrangler vars are strings). Add the same keys with conservative defaults to cloudflare/wrangler.jsonc vars (:6) and env.staging.vars (:66) so staging can override. Suggested defaults: chat/ai_query 10/min anon, 60/min authed; recall 20/120; mcp 30/300; directory 30 anon.

6. Cron cleanup (optional, cheap). In processDueSchedules (cron */30, worker.ts:540) add a DELETE FROM contextmem_rate_limits WHERE window_start < ? (older than ~1h) so the table doesn't grow unbounded.

7. Docs. Document the new vars + behavior in .env.example (root) and README.md; note the optional CF ratelimit binding stub. Tick the relevant line in docs/launch-checklist.md (D5 abuse-protection precondition for going public).

New / changed files

• apps/api/migrations/0007_rate_limits.sql (new)
• apps/api/src/worker.ts (helpers + router guards + cors/WorkerEnv edits)
• apps/api/cloudflare/wrangler.jsonc (vars top-level + env.staging)
• apps/api/cloudflare/wrangler.example.jsonc (mirror the new vars)
• apps/api/src/worker.test.ts (new tests + MemoryD1 mock support)
• .env.example, README.md, docs/launch-checklist.md (docs)

Test plan

• Test-infra prerequisite (gotcha): MemoryD1Database (worker.test.ts:845) only answers enumerated queries; unknown queries return null/empty, so the rate-limit upsert would never count. Add a contextmem_rate_limits map + handle the INSERT ... ON CONFLICT ... RETURNING count in MemoryD1Statement.first() (worker.test.ts:878) returning { count }, incrementing per bucket_key.
• Unit (vitest, worker.test.ts), using vi.useFakeTimers()/vi.setSystemTime to pin the window:
  1. Under limit passes — N < limit requests to /api/memwal/chat return non-429 and a decreasing X-RateLimit-Remaining.
  2. Over limit 429s — the (limit+1)th anon request returns 429 with Retry-After header and body code RATE_LIMITED.
  3. Authed bucket > anon — set RATE_LIMIT_CHAT_PER_MIN low; exhaust as anon (no headers) → 429; same count with x-memwal-account-id+x-memwal-authorization delegate headers still passes (uses _AUTHED_ limit).
  4. Per-key not global — exhaust IP A (cf-connecting-ip: 1.1.1.1) → 429; IP B (2.2.2.2) still 200. Repeat for two distinct read tokens on /mcp.
  5. Window reset — advance system time past windowSec; a blocked key passes again.
  6. MCP shape — over-limit /mcp returns JSON-RPC 429 (mcpJsonError) not the REST body.
• Manual / staging — for i in $(seq 1 40); do curl -s -o /dev/null -w "%{http_code} " https://contextmem-backend-staging.petlofi.workers.dev/api/memwal/chat -d '{"message":"hi"}' -H 'content-type: application/json'; done shows 200s then 429s; inspect X-RateLimit-Remaining counting down and Retry-After on the 429. Confirm SSRF guard isPrivateIpv4 (worker.ts:4288) and /api/demo/extractions 1/day quota still behave (no regression).

Risks & gotchas

• MemoryD1 mock must be extended or every rate-limit test silently passes (count always 0). Highest-leverage gotcha.
• CORS expose-headers — without step 4, browser clients see the 429 but can't read X-RateLimit-*.
• D1 write on every public read adds a write + latency to hot paths; window is short and counts are tiny, but note it. The CF binding is the future escape hatch for the hottest unauth path.
• Fixed-window burst — up to 2× limit across a boundary; acceptable for abuse protection (note sliding-window as a later upgrade).
• IP trust — clientIp prefers cf-connecting-ip (trustworthy behind CF); the x-forwarded-for fallback is spoofable but only hit off-CF (local/tests).
• MCP client tolerance — return HTTP 429 + Retry-After; mcp-remote may not surface the body, so status + header carry the signal.
• Don't throw statusError for 429 — the central catch in handleWorkerRequest (worker.ts:565) rebuilds the body via json() and would drop Retry-After/X-RateLimit-*; build the Response directly.
• Don't double-count — enforce only at the router; keep consumeDemoQuota independent (durable daily quota vs ephemeral window).
• Migration is manual on this repo (Worker not auto-deployed) — apply 0007 to prod + staging D1.

Acceptance-criteria mapping

• rateLimit(env, key, {limit, windowSec}) → {allowed, remaining, resetAt} via D1 sliding/fixed window mirroring consumeDemoQuota → Step 2 (D1 chosen over CF binding for testability + remaining/resetAt; rationale in Approach).
• Distinct anon (per-IP) vs authed (per read-token/account) buckets, higher authed limits → Steps 2 (rateLimitKey) + 5 (separate _AUTHED_ vars); test fix(ci): force bun + pin wrangler in Deploy workflow #3.
• Enforced on /mcp, /api/memwal/chat, aiQueryRun, /api/memwal/recall, /api/directory → Step 3 (all five wired).
• 429 with Retry-After + typed RATE_LIMITED code; X-RateLimit-Remaining/-Reset on success → Step 2 (rateLimited429/withRateLimitHeaders) + Step 4 (expose headers); tests Walrus storage + Memory via Tatum, plus CI deploy #1, Memory knowledge graph: 3D neural constellation + grounded entity panel #2, Build HarborClient: reserve→sign→finalize bucket, multipart upload, status poll, download #6.
• Env-configurable via wrangler.jsonc vars with safe defaults, staging override → Step 5.
• Tests: under-limit pass, over-limit 429, authed>anon, per-key not global → Test plan unit Walrus storage + Memory via Tatum, plus CI deploy #1–docs: add product roadmap (Harbor storage + Talus on-chain direction) #4 (+ Spike: run @mysten/seal encrypt+decrypt inside a Cloudflare Worker #5/Build HarborClient: reserve→sign→finalize bucket, multipart upload, status poll, download #6).
• Implementation notes: keep separate from usage ledger, don't regress SSRF guard → Step 1 (own table) + Steps 3/Risks (isPrivateIpv4 untouched); optional rate_limited ledger event deferred (no ledger exists yet — issue's usage-ledger item is separate).

Effort
M — ~2 person-days. ~0.5d helper + router wiring + config, ~0.5d MemoryD1 mock + tests, ~0.5d docs + staging deploy/verify, ~0.5d buffer for tuning defaults and the MCP 429 path.

---

Implementation plan generated for execution (planning phase). Part of the roadmap (#4).

[roblambert9]

Hi, I read issue #11 on contextMeM. You wrote that the token-less public MCP reads and LLM endpoints "are completely uncapped: an open cost/abuse vector that blocks flipping the repo public," and that you want per-IP and per-token limits so a single client can't run up unbounded Workers AI / OpenRouter cost.

Rate limiting stops the bleeding. The adjacent question is whether you want those same per-token reads to eventually meter and bill per call instead of just 429ing, since you already have the read-token and account buckets you describe.

I run a small, fixed-scope thing for exactly that gap: a $99 MCP Tool Metering Readiness Audit. It is async, takes me about one to two days, and for your worker.ts shape it gives you: where a per-call meter and a tamper-evident SHA-256 receipt would sit relative to your existing consumeDemoQuota and ledger split, a pricing shape for the /mcp reads and the chat/recall endpoints, a runnable PAPER_MODE demo wired to your endpoint shape so no real money moves, and a copy-pasteable go-live plan. It is built on an MIT SDK I maintain (nano-empire-tollbooth).

Full honesty so there is no confusion: the SDK has earned $0 so far and runs in paper mode. I am selling readiness, not revenue, and I am not claiming live payments or traction. The only thing I can prove today is a reproducible dogfood run where an external agent hits a 402, pays a 1 cent toll through a mock verifier, escrow locks then releases, and 33 tests pass. That proves the protocol mechanics, not real money changing hands.

Repo: github.com/roblambert9/nano-empire-tollbooth · PyPI: pypi.org/project/nano-empire-tollbooth

If it's not useful, no worries, I won't follow up.