Define the canonical extract / recall / remember MCP triad as the agent interface

[harrymove-ctrl]

Context

The local MCP server (packages/mcp/src/index.ts) exposes ~20 fine-grained tools (scrape_markdown, crawl_site, extract_brand, remember_site_snapshot, upload_proof_to_walrus, recall_site_context, query_site_memory, get_context_bundle, ...) and the hosted server (packages/mcp/src/hosted.ts) exposes a different read-only set via registerToolAliases. There is no single, stable triad an agent (or a Talus/Nexus node) can bind to. The roadmap (exec summary + §6) makes contextmem.extract / .recall / .remember the canonical capabilities to publish as Talus Tools. Note buildAgentReadableSite is already imported in index.ts but never wired into a tool.

Goal / user story

As an agent builder, I want three high-level, well-documented MCP tools — contextmem_extract, contextmem_recall, contextmem_remember — with stable typed I/O, so I can integrate ContextMEM in one step and have my calls map 1:1 to Talus Nexus DAG nodes, without learning 20 internal verbs.

Acceptance criteria

• contextmem_extract accepts { target, mode?: 'auto'|'web'|'walrus', maxPages?, namespace? } and returns a stable JSON envelope { namespace, runId, target, pageCount, chunkGraphDigest, artifactDigest, summary } by composing crawlWebSite/scrapeWebPage/resolveWalrusTarget + buildAgentReadableSite + facts.
• contextmem_remember accepts { runDir|runId, namespace?, storeOnWalrus?: boolean, whatChanged? } and unifies today's remember_site_snapshot + upload_proof_to_walrus + rememberStorageIndex, returning { namespace, writeMode, written, receipt?, memoryRef }.
• contextmem_recall accepts { namespace, query, topK?, includeSources?: boolean } and returns { namespace, query, answer?, sources: [...], memory }, unifying recall_site_context / query_site_memory / get_context_bundle.
• All three tools exist on BOTH the local (index.ts) and hosted (hosted.ts) servers with identical names and output shapes (hosted extract/remember may be gated/no-op where the Worker can't pack — return a typed unsupported envelope rather than throwing).
• The ~20 existing tools remain registered as advanced aliases (reuse the registerToolAliases pattern) so nothing breaks, but tool descriptions point agents to the triad first.
• A shared zod schema module exports the three input/output types so cli, mcp, and any Nexus tool-def import the same contract; unit tests assert the output envelope shape for each tool.

Implementation notes

• Touch packages/mcp/src/index.ts, packages/mcp/src/hosted.ts, and add packages/mcp/src/triad.ts (or packages/core if shared with CLI) holding the zod I/O schemas + the composition functions.
• MCP tool names cannot contain dots; use contextmem_extract|recall|remember as the MCP names and reserve the dotted contextmem.extract form for the Talus/Nexus node identifiers (document the mapping).
• extract should reuse createRunId + the run-dir layout already produced (manifest.json, llms.txt), and surface chunkGraphDigest()/artifactDigest (from packProofBundle) since those are the verifiable anchors the receipt hook (separate issue) needs.
• remember storage path is Node-only today (packProofBundle shells tar); on the hosted/Worker side return { storedOnWalrus: false, reason } until the Worker-side R2 zip packer lands — keep the contract identical so the receipt hook is uniform.
• recall on hosted should delegate to the existing store.recallMemory + searchContext plumbing already in hosted.ts.
• Keep output JSON-only (the text()/textResult() helpers) so Nexus can parse deterministically.

Sui Overflow angle

This triad is the agent-facing surface ContextMEM demos for Sui Overflow: a single, legible extract → remember → recall loop is what a Talus/Nexus DAG composes, and what the on-chain attribution receipt attaches to. Judges and agent builders see one clean Sui-native memory primitive instead of 20 verbs, and every node output already carries the digests that prove what was extracted/stored on Walrus.

Dependencies

None (foundational). Blocks: the on-chain attribution-receipt issue and the reference-agent-templates issue.

Part of the ContextMEM roadmap (#4) • Sui Overflow build.

[harrymove-ctrl]

Implementation Plan

Approach
Introduce a single canonical agent triad — contextmem_extract / contextmem_recall / contextmem_remember — as thin, well-documented MCP tools that wrap the existing fine-grained verbs already proven in packages/cli/src/index.ts and packages/mcp/src/index.ts. The key design decision is a two-layer split: (1) a runtime-agnostic, zod-only contract module in @contextmem/core (triad-schema.ts) that cli, mcp (local + hosted), the Worker, and future Nexus tool-defs all import as the single source of truth; and (2) Node-only composition functions in packages/mcp/src/triad.ts that orchestrate crawlWebSite/scrapeWebPage/resolveWalrusTarget → buildAgentReadableSite (+ facts) → packProofBundle/uploadProofBundle → MemWalMcpClient. The ~20 existing tools stay registered as advanced aliases; their descriptions are reworded to point agents at the triad first. Hosted extract/remember return a typed unsupported envelope (the Worker can't shell tar) while hosted recall composes the existing store.recallMemory + searchContext plumbing — keeping output shapes identical across both servers so the downstream receipt hook is uniform.

Prerequisites & dependencies

• Blocking issues: none. Issue Define the canonical extract / recall / remember MCP triad as the agent interface #15 is foundational ("Dependencies: None"). It blocks the on-chain attribution-receipt issue and the reference-agent-templates issue, so the envelope must surface chunkGraphDigest + artifactDigest (the receipt anchors).
• Open roadmap decision (non-blocking for this issue): the dotted contextmem.extract|recall|remember Nexus node identifiers are reserved/documented only; the actual Nexus tool-def wrapper is a later issue (roadmap §2/§6). Confirm the Nexus schema/ABI before freezing struct names, but the MCP I/O contract can ship now.
• Creds/bindings (no new ones required):
  • Local remember/recall: MEMWAL_PRIVATE_KEY (Ed25519 hex seed) + MEMWAL_ACCOUNT_ID (already required by MemWalMcpClient, see packages/memwal/src/index.ts:88-89).
  • Local remember with storeOnWalrus:true: TATUM_API_KEY (already used by uploadProofBundle).
  • Hosted recall: existing per-request MemWal headers/secrets via maybeWithMemWalRecall (apps/api/src/worker.ts:3609). No Vectorize/Harbor/Sui wallet needed for this issue.
• New dep: add "zod" to packages/core/package.json dependencies (root already pins zod ^4.2.0; mcp already uses it).

Step-by-step

1. Create the shared contract module packages/core/src/triad-schema.ts.

   • Define zod schemas + inferred TS types:
     • ExtractInput = { target: string; mode?: 'auto'|'web'|'walrus'; maxPages?: number; namespace?: string }
     • ExtractOutput = { namespace; runId; target; pageCount; chunkGraphDigest; artifactDigest; summary }
     • RememberInput = { runDir?: string; runId?: string; namespace?: string; storeOnWalrus?: boolean; whatChanged?: string }
     • RememberOutput = { namespace; writeMode: 'snapshot'|'storage'|'snapshot+storage'|'unsupported'; written: number; receipt?: WalrusStorageReceipt; memoryRef?: {...}; supported: boolean; reason?: string }
     • RecallInput = { namespace: string; query: string; topK?: number; includeSources?: boolean }
     • RecallOutput = { namespace; query; answer?; sources: Array<{...}>; memory }
   • Export a TRIAD_TOOL_NAMES = { extract: 'contextmem_extract', recall: 'contextmem_recall', remember: 'contextmem_remember' } constant and a NEXUS_NODE_IDS = { extract: 'contextmem.extract', ... } mapping (document that MCP names cannot contain dots; the dotted form is the Talus/Nexus node id only).
   • Export an unsupportedRemember(namespace, reason) helper that returns the typed unsupported envelope.
   • Re-export from packages/core/src/index.ts (append export * from "./triad-schema.js";). Add the bare zod import; do not import walrus/memwal here (core is the lowest layer — keep it dependency-free upward).

2. Create the local composition module packages/mcp/src/triad.ts (Node-only; imports core+walrus+memwal which mcp already depends on).

   • extractContext(input, { runsDir }): Promise<ExtractOutput>:
     • Parse with ExtractInputSchema. Resolve mode: 'walrus' (or 'auto' + 0x/preview target) → resolveWalrusTarget + materializeWalrusSite (mirror index.ts:120-131); else 'web' → crawlWebSite(target, { maxPages, maxDepth: 1, includeImages: true }) (mirror CLI index.ts:395) or scrapeWebPage for single-page.
     • const runId = createRunId('mcp-extract') (utils.ts:10); outputDir = path.resolve(runsDir, runId).
     • Build facts via buildSiteFacts(target, pages, buildChunks(pages)) (facts.ts:374) wrapped in try/catch (LLM optional).
     • Call buildAgentReadableSite({ runId, target, outputDir, pages, facts, ...walrus }) (package.ts:61). This already writes context/manifest.json, llms.txt, chunks.ndjson, and context/snapshot.json (which carries artifactDigest + chunkGraphDigest, package.ts:153-162).
     • Read context/snapshot.json for artifactDigest/chunkGraphDigest; compute namespace = input.namespace ?? namespaceForTarget(...) (utils.ts:123); summary = summarizeSnapshot(...) (memwal/index.ts:302). Return validated ExtractOutput.
   • rememberContext(input, { runsDir }): Promise<RememberOutput>:
     • Resolve absRunDir from runDir or path.resolve(runsDir, runId); read context/manifest.json. Resolve namespace like index.ts:212-213.
     • Always do the semantic snapshot write: build SiteSnapshot from manifest (mirror index.ts:170-182) → rememberSnapshot → counts as written.
     • If storeOnWalrus: packProofBundle(absRunDir) (storage.ts:231) → uploadProofBundle({...artifactDigest, wait:true}) (storage.ts:183) → write context/storage.json → rememberStorageIndex({ namespace, target, runId, receipt, whatChanged }) (memwal/index.ts:164); set receipt, memoryRef, writeMode='snapshot+storage'.
     • Else writeMode='snapshot'. Return validated RememberOutput with supported:true.
   • recallContext(input): Promise<RecallOutput>:
     • Parse; memory = await new MemWalMcpClient().recallSiteContext(namespace, query) (memwal/index.ts:171). If includeSources, also call analyzeSiteMemory (memwal/index.ts:175) and map into sources. Return validated RecallOutput.

3. Register the triad on the local server packages/mcp/src/index.ts.

   • Add three server.tool(...) registrations at the top (before scrape_markdown) using TRIAD_TOOL_NAMES, the zod input schemas from core, rich descriptions ("Canonical agent interface — start here. Maps to Nexus node contextmem.extract"), delegating to the triad.ts functions with { runsDir }. Keep returning JSON via the existing text() helper (index.ts:313).
   • Reword the 20 existing tool descriptions to prefix "Advanced/low-level — prefer contextmem_extract|recall|remember." (no behavior change, only .describe()/description strings).

4. Register the triad on the hosted server packages/mcp/src/hosted.ts.

   • In createHostedContextMemMcpServer, register all three via the existing registerToolAliases pattern (hosted.ts:91/270), names contextmem_extract|recall|remember, sharing namespaceInputSchema.
   • contextmem_recall: compose resolveNamespace → store.searchContext (or fallback searchHostedArtifacts) for sources → store.recallMemory?.(namespace, query) for memory (mirror get_context_bundle at hosted.ts:241-249), shaped to RecallOutput (set memory.available:false reason when MemWal headers absent).
   • contextmem_extract and contextmem_remember: return the typed unsupported envelope via textResult(unsupportedRemember(namespace, "Hosted Worker cannot pack/upload — use the local @contextmem/mcp server or CLI")) — not an MCP error, so Nexus can parse it deterministically. (Wiring an enqueue path to processExtractionJob at worker.ts:1951 is explicitly deferred to the Worker-side R2 zip-packer roadmap item.)

5. Point CLI at the shared contract (lightweight): import the triad types from @contextmem/core in packages/cli/src/index.ts so any future contextmem extract|remember|recall CLI verbs reuse the identical I/O — satisfies "cli imports the same contract" without rewriting the existing inline composition now. (Optional: thin CLI subcommands that call triad.ts — note as a follow-up, not required by ACs.)

6. Tests — see Test plan. Add packages/mcp/src/triad.test.ts and extend packages/mcp/src/hosted.test.ts.

New / changed files

• NEW packages/core/src/triad-schema.ts — zod I/O schemas, types, TRIAD_TOOL_NAMES, NEXUS_NODE_IDS, unsupportedRemember.
• NEW packages/mcp/src/triad.ts — extractContext / rememberContext / recallContext composition.
• NEW packages/mcp/src/triad.test.ts — envelope-shape unit tests.
• EDIT packages/core/src/index.ts — re-export triad-schema.
• EDIT packages/core/package.json — add zod dependency.
• EDIT packages/mcp/src/index.ts — register 3 triad tools; reword 20 descriptions.
• EDIT packages/mcp/src/hosted.ts — register 3 triad tools (recall composed; extract/remember unsupported).
• EDIT packages/mcp/src/hosted.test.ts — assert hosted triad output shapes.
• EDIT (light) packages/cli/src/index.ts — import shared triad types.

Test plan

• Unit (core): zod round-trip — valid inputs parse, bad inputs reject; ExtractOutput/RememberOutput/RecallOutput .safeParse() accept the exact envelopes; unsupportedRemember() matches RememberOutputSchema with supported:false, writeMode:'unsupported'.
• Unit (mcp local, triad.test.ts): mock crawlWebSite/buildSiteFacts/MemWalMcpClient/packProofBundle/uploadProofBundle (vitest vi.mock); assert extractContext returns all 7 keys with a real runId (mcp-extract_…), pageCount === pages.length, and chunkGraphDigest/artifactDigest read from the generated context/snapshot.json; assert rememberContext returns writeMode:'snapshot' without storeOnWalrus and 'snapshot+storage' (with receipt+memoryRef) when true; assert recallContext returns {namespace,query,sources,memory}.
• Unit (mcp hosted): extend hosted.test.ts — instantiate the server with the existing mock HostedNamespaceStore; assert contextmem_extract/remember produce a parseable unsupported envelope (not isError), and contextmem_recall returns the RecallOutput shape (sources from searchContext, memory.available:false when no recallMemory).
• Integration / manual (proves it works):
  • Real round-trip with creds set: extractContext({target:'https://example.com'}) → feed its runDir into rememberContext({runId, storeOnWalrus:true}) → assert a real Tatum/Walrus WalrusStorageReceipt with certified:true + a blobId, then recallContext({namespace, query}) returns non-empty memory — closing the extract→remember→recall loop.
  • MCP smoke: run the stdio server (tsx packages/mcp/src/index.ts), list tools, confirm the triad appears first and the 20 advanced tools still register.
• CI: bun-check.yml (typecheck + test + build) must stay green; verify @contextmem/core typechecks with the new zod import.

Risks & gotchas

• Worker can't pack: packProofBundle shells tar via child_process (storage.ts), unavailable under nodejs_compat. Hosted extract/remember MUST return the typed unsupported envelope, never throw — identical contract so the receipt hook is uniform. Do not attempt Worker-side encryption/zip here (separate roadmap item).
• MCP name constraint: tool names cannot contain dots — use underscores; reserve dotted names for Nexus only. Document the mapping in the schema module.
• Two artifactDigests: snapshot.json.artifactDigest (sha256 over per-file path:sha256, snapshot.ts:100) vs packProofBundle.artifactDigest (sha256 over the .tgz bytes, storage.ts:243). extract surfaces the snapshot digest (pre-upload); remember's receipt.artifactDigest is the tgz digest. Label clearly so the receipt issue anchors the right one.
• JSON-only outputs: keep using text()/textResult() so Nexus parses deterministically; never return prose.
• MemWal flakiness: recall/remember depend on the relayer (already retried in memwal/index.ts:254); missing MEMWAL_* throws in the constructor — guard local remember/recall so a missing-cred error becomes a clean message, and hosted recall degrades to memory.available:false.
• No regressions: all 20 tools must remain registered (only descriptions change); the hosted restore_site_memory/recall_memory aliases stay intact.
• Network split (mainnet/testnet): out of scope here — remember keeps today's Tatum/mainnet path; do not reconcile Harbor/testnet in this issue.
• Backward compat: rememberStorageIndex text format is unchanged (no schema migration); no D1 changes.

Acceptance-criteria mapping

• contextmem_extract envelope → Steps 1 (schema), 2 (extractContext), 3 (local register), 4 (hosted unsupported).
• contextmem_remember unifies snapshot+upload+index → Steps 1, 2 (rememberContext), 3, 4.
• contextmem_recall unifies recall/query/bundle → Steps 1, 2 (recallContext), 3, 4 (hosted compose of recallMemory+searchContext).
• Both servers, identical names/shapes; hosted extract/remember return typed unsupported → Steps 3 + 4 + unsupportedRemember.
• 20 tools remain as advanced aliases; descriptions point to triad first → Step 3 (local rewording) + hosted registerToolAliases untouched.
• Shared zod module imported by cli/mcp/Nexus + unit tests assert envelope shape → Steps 1 (core module), 5 (cli import), 6 + Test plan (unit tests).

Effort: M, ~3 person-days. ~1d schema + local composition (mostly re-wiring proven CLI/MCP code), ~0.5d hosted wiring, ~0.5d description rewording, ~1d tests + a real extract→remember→recall round-trip.

---

Implementation plan generated for execution (planning phase). Part of the roadmap (#4).