3D memory constellation usability, a11y & performance pass (reduced-motion, WebGL fallback, keyboard, mobile)

[harrymove-ctrl]

Context

NamespaceMemoryConstellation.tsx always autorotates (controls.autoRotate = true, line 345), runs a permanent requestAnimationFrame loop (line 362) with UnrealBloomPass/EffectComposer, and caps setPixelRatio(min(dpr, 2)) (line 333). It ignores prefers-reduced-motion, has no WebGL-unavailable fallback, no keyboard navigation, no Esc-to-deselect (DOM labels are aria-hidden, line 560), and is crowded on mobile. The landing hero already respects reduced-motion (main.tsx:953), so the pattern exists but isn't applied here.

Goal / user story

As a keyboard, low-power, reduced-motion, or mobile user, I can use the memory graph without motion sickness or a frozen/blank canvas — with a usable 2D fallback when WebGL is unavailable.

Acceptance criteria

• Autorotate + bloom + the RAF loop are gated on prefers-reduced-motion (static/paused render when the user prefers reduced motion).
• When WebGL is unavailable, a 2D/list fallback renders from facts-to-graph.ts (+ related-neighbors.ts) instead of a blank canvas.
• Esc deselects the active node and exits fullscreen.
• Search results and nodes are keyboard-navigable (focus + arrow/enter, aria-activedescendant), not mouse-only.
• On small viewports (<~640px) pixelRatio and bloom strength are reduced, and the graph + detail panel are usable (verified at phone width).
• Switching to the fallback disposes the renderer/composer (no leaked WebGL context).

Implementation notes

• Reuse the window.matchMedia('(prefers-reduced-motion: reduce)') pattern from main.tsx:953; store it in a ref and branch the animate() loop / controls.autoRotate in NamespaceMemoryConstellation.tsx.
• Add a WebGL capability check before new THREE.WebGLRenderer; fall back to a DOM graph using the already-imported facts-to-graph.ts and related-neighbors.ts.
• Add a keydown handler for Escape; make MemorySearchBox results a keyboard listbox.
• Mobile: lower UnrealBloomPass strength and setPixelRatio(1) under the breakpoint; consider a bottom-sheet detail panel.
• Gotcha: dispose renderer/composer/geometries on fallback switch and on unmount to avoid context-loss leaks.

Sui Overflow angle

The 3D "memory constellation" is the signature visual for the Walrus-backed knowledge graph — the thing judges screenshot. Making it reduced-motion-safe (for clean screen recordings), WebGL-resilient, and mobile-friendly keeps the wow-factor working across judges' varied devices.

Dependencies

None — the 2D fallback uses the existing facts-to-graph.ts.

Part of the ContextMEM roadmap (#4) • Sui Overflow build.

[harrymove-ctrl]

Implementation Plan

Approach
Split the monolithic NamespaceMemoryConstellation.tsx (575 lines) into a thin parent that owns shared chrome (search box, legend, fullscreen button, detail panel, selected state) and chooses a renderer, plus a Constellation3D child that owns the THREE scene, and a new MemoryGraph2D child (DOM/SVG list) used when WebGL is unavailable. This makes the "dispose on fallback switch" guarantee free (React unmounts the 3D child → its existing cleanup runs), keeps hooks unconditional, and isolates the WebGL/reduced-motion/mobile logic. Reduced-motion is read via a live matchMedia ref (reusing the main.tsx:953 + staggered-text.tsx:129-141 patterns) and branches the animate loop / controls.autoRotate / bloom; on reduced-motion or static state we render on-demand (controls change + resize) instead of a permanent requestAnimationFrame loop.

Prerequisites & dependencies

• No blocking issues — the issue states "Dependencies: None"; the 2D fallback reuses the already-imported facts-to-graph.ts (MemoryGraph prop is already passed in) and related-neighbors.ts. This is roadmap §4 P0 ("Reduced-motion + WebGL/low-power fallback for the constellation").
• No new creds/bindings — pure client-side React/three work in apps/web. No Worker/D1/Harbor/Sui touch.
• Libs already present: three@0.184 (incl. three/examples/jsm/capabilities/WebGL.js), motion/react (exports useReducedMotion), lucide-react, @testing-library not installed but jsdom@27 + vitest@4 are available at the root (env is node; use a // @vitest-environment jsdom pragma for DOM-touching tests).
• Test runner: root vitest run over apps/**/*.test.ts (env node). Existing sibling tests: facts-to-graph.test.ts, related-neighbors.test.ts, entity-detail.test.ts.

Step-by-step

1. Add capability/preference helpers — new apps/web/src/components/namespace-memory/capabilities.ts:

   • webglSupported(): boolean — create a throwaway <canvas> and test getContext('webgl2') || getContext('webgl') inside a try/catch (or wrap three/examples/jsm/capabilities/WebGL.js's isWebGLAvailable). DOM-touching → covered by a jsdom-pragma test.
   • prefersReducedMotion(): boolean — window.matchMedia('(prefers-reduced-motion: reduce)').matches (mirror main.tsx:953).
   • isCompactViewport(): boolean — window.matchMedia('(max-width: 640px)').matches.
   • Pure reducer nextActiveIndex(current, key, length) for arrow/Home/End keyboard nav (unit-testable in node).

2. Refactor the parent NamespaceMemoryConstellation (NamespaceMemoryConstellation.tsx):

   • Lift selected/setSelected and the search/legend/panel/fullscreen JSX (currently lines 548-574) into the parent so they work in both render modes.
   • Add state: webglOk (init webglSupported()), reducedMotion, compact; subscribe to both media queries with the addEventListener('change', …) pattern from staggered-text.tsx:129-141 (update state on change).
   • Add a single keydown handler on rootRef (replaces the missing Esc support): if focus is in the search input let the search box own Esc (clear query); else Escape → if selected deselect (setSelected(null)), else if fullscreen call toggleFullscreen(). (Keep existing toggleFullscreen at lines 286-295 and the fullscreen change effect at 273-284.)
   • Render {webglOk ? <Constellation3D … /> : <MemoryGraph2D … />}. Pass graph, selected, onSelect={setSelected}, reducedMotion, compact, and onContextLost={() => setWebglOk(false)}.

3. Create Constellation3D.tsx — move the THREE setup verbatim from the current one-time scene effect (lines 322-453), the selected-autoRotate effect (456-459), and the mesh-rebuild effect (462-539), then apply the gating:

   • Reduced-motion gate (AC Walrus storage + Memory via Tatum, plus CI deploy #1): keep reducedMotion in a ref updated each render. In the scene effect, set controls.autoRotate = !reducedMotionRef.current (replaces hard autoRotate = true at line 345). Replace the unconditional s.raf = requestAnimationFrame(animate) (line 362): when reduced-motion, do not schedule the loop — instead define renderOnce() (freeze uTime at a fixed value, run controls.update() + composer.render() / direct renderer.render, then reproject labels) and call it on controls.addEventListener('change', renderOnce) and from the ResizeObserver. When motion is allowed, keep the existing RAF loop. Gate bloom: under reduced-motion either skip composer.addPass(bloom) (render via RenderPass+OutputPass only) or set bloom.strength = 0. Re-evaluate when the media query flips (re-run effect via reducedMotion dep, or toggle inside a small effect).
   • WebGL context loss (AC Memory knowledge graph: 3D neural constellation + grounded entity panel #2/Build HarborClient: reserve→sign→finalize bucket, multipart upload, status poll, download #6): add renderer.domElement.addEventListener('webglcontextlost', e => { e.preventDefault(); onContextLost(); }) so a runtime loss flips the parent to the 2D fallback.
   • Mobile perf (AC Spike: run @mysten/seal encrypt+decrypt inside a Cloudflare Worker #5): replace setPixelRatio(Math.min(devicePixelRatio, 2)) (line 333) with setPixelRatio(compact ? 1 : Math.min(devicePixelRatio, 2)); construct UnrealBloomPass (line 349, currently strength 1.6) with reduced strength/radius under compact (e.g. compact ? 0.6 : 1.6). Recompute both in the ResizeObserver (lines 431-438) so rotation/resize across the breakpoint updates them.
   • Focus-on-select: instead of the parent calling s.focusRec, have the child watch the selected prop in an effect → fly the camera (existing focusRec, lines 423-428). Under reduced-motion, jump the camera/target directly (set, don't lerp via s.desired) then renderOnce(), since there's no RAF loop to run the lerp.
   • Dispose composer (AC Build HarborClient: reserve→sign→finalize bucket, multipart upload, status poll, download #6): extend the cleanup (lines 442-452) — it currently disposes meshes/stars/controls/renderer but not the composer/bloom. Add composer.dispose() and bloom.dispose() (and remove controls/canvas listeners) so unmounting to the fallback leaks no GL context or render targets.
   • Keep the projected DOM labels host but make it keyboard/SR-friendly (see step 5).

4. Create MemoryGraph2D.tsx (AC Memory knowledge graph: 3D neural constellation + grounded entity panel #2) — a no-WebGL fallback rendered from the same MemoryGraph prop (output of factsToMemoryGraph, facts-to-graph.ts) plus relatedNeighbors (related-neighbors.ts):

   • A grouped, scrollable list/SVG of nodes sorted by salience, colored via colorForType/labelForType from lib/entity-colors.ts (signatures at entity-colors.ts:87/:91), each row a focusable <button> (roving tabindex, arrow/Enter nav using nextActiveIndex) calling onSelect(node). Selecting reuses the same shared MemoryChunkPanel/relatedNeighbors path as 3D.
   • Render inside the existing .nmc-overlay-msg/.nmc-canvas box so layout/fullscreen still work; add styles in namespace-memory.css.

5. Make search + nodes keyboard-navigable (AC docs: add product roadmap (Harbor storage + Talus on-chain direction) #4) — MemorySearchBox.tsx (currently a plain input + button list, 31 lines):

   • Input → role="combobox", aria-expanded, aria-controls="nmc-results", aria-autocomplete="list", aria-activedescendant = active option id; results container → role="listbox" id="nmc-results"; each result → role="option" with stable id + aria-selected.
   • Add onKeyDown: ArrowUp/Down move active index (nextActiveIndex), Enter selects the active option (onSelect + clear), Esc clears query. Mouse onClick still works.
   • For 3D nodes specifically: add a visually-hidden focusable node list (or remove aria-hidden="true" from the labels host at line 560 and make labels focusable buttons that call onSelect) so screen-reader/keyboard users can reach nodes without a pointer. The 2D fallback already satisfies node keyboard-nav inherently.

6. CSS (namespace-memory.css):

   • Style the combobox/listbox active option ([aria-selected="true"]) and focus rings on .nmc-results button, .nmc-related, fallback rows.
   • Mobile bottom-sheet (AC Spike: run @mysten/seal encrypt+decrypt inside a Cloudflare Worker #5): add @media (max-width: 640px) for .nmc-panel (currently top:0;right:0;height:100% at lines 23-28) → left:0;right:0;bottom:0;top:auto;height:auto;max-height:75vh;border-left:none;border-top:1px solid …;border-radius:12px 12px 0 0. Tighten .nmc-search/.nmc-legend for small widths.
   • Style MemoryGraph2D rows/groups (reuse --cm-* tokens; keep brand lime accents).

7. Call-site sanity — no change needed at FactsPanel (main.tsx:7253, constellation rendered in .nmc-host at main.tsx:7358-7360); the component keeps the same { graph, entityDetail } props. Verify constellationGraph (main.tsx:7260) still feeds both modes.

New / changed files

• apps/web/src/components/namespace-memory/capabilities.ts (new)
• apps/web/src/components/namespace-memory/Constellation3D.tsx (new — extracted THREE scene)
• apps/web/src/components/namespace-memory/MemoryGraph2D.tsx (new — 2D/list fallback)
• apps/web/src/components/namespace-memory/NamespaceMemoryConstellation.tsx (changed — thin parent: capability/RM/compact state, Esc handler, mode switch, shared chrome)
• apps/web/src/components/namespace-memory/MemorySearchBox.tsx (changed — combobox/listbox a11y + keyboard nav)
• apps/web/src/components/namespace-memory/namespace-memory.css (changed — listbox/focus styles, mobile bottom-sheet, 2D fallback rows)
• apps/web/src/components/namespace-memory/capabilities.test.ts + MemorySearchBox.keyboard.test.ts (new tests)

Test plan

• Unit (node env): nextActiveIndex wrap/clamp/Home/End; search filter/ranking still matches MemorySearchBox haystack behavior. Reuse existing facts-to-graph.test.ts/related-neighbors.test.ts to confirm the 2D fallback consumes the same graph shape.
• Unit (jsdom pragma): webglSupported() returns false when getContext throws/returns null; MemorySearchBox keyboard: typing → listbox opens with aria-activedescendant, ArrowDown/Enter fires onSelect, Esc clears.
• Integration: render NamespaceMemoryConstellation with a stubbed webglSupported → false and assert the MemoryGraph2D list renders the expected nodes (no canvas, no THREE import path hit) — proves AC Memory knowledge graph: 3D neural constellation + grounded entity panel #2.
• Manual (proves it works): (a) DevTools → Rendering → "Emulate prefers-reduced-motion: reduce": confirm no autorotate, no continuous RAF (Performance panel shows idle, not a steady 60fps loop), bloom off/reduced, scene still visible and orbit-draggable. (b) chrome://gpu disable WebGL (or stub) → 2D list renders, and in DevTools Memory/Performance the WebGL context count drops to 0 after switch (AC Build HarborClient: reserve→sign→finalize bucket, multipart upload, status poll, download #6 — no leaked context). (c) Esc deselects an open node and exits fullscreen (AC fix(ci): force bun + pin wrangler in Deploy workflow #3). (d) Tab to search, arrow/enter to select; Tab through 2D list (AC docs: add product roadmap (Harbor storage + Talus on-chain direction) #4). (e) Resize to 375px width: pixelRatio 1, reduced bloom, detail panel as bottom-sheet, graph usable (AC Spike: run @mysten/seal encrypt+decrypt inside a Cloudflare Worker #5).
• bun run -C apps/web typecheck + root vitest run green; vite build succeeds.

Risks & gotchas

• No leaked GL context: the win from the parent/child split is that unmounting Constellation3D runs cleanup — but only if cleanup is complete. The current cleanup (lines 447-451) misses composer.dispose()/bloom.dispose(); add them or the fallback switch still leaks render targets (directly violates AC Build HarborClient: reserve→sign→finalize bucket, multipart upload, status poll, download #6). Also remove all listeners (controls change, webglcontextlost, pointer down/up).
• Reduced-motion without an RAF loop: camera fly (s.desired lerp) and click ripple pulses depend on the loop. Under reduced-motion, jump the camera directly and accept static pulses; render on controls 'change' + resize so dragging/resizing still repaint (otherwise the canvas appears frozen after the first user input).
• State that must survive without re-mounting the scene: put reducedMotion/compact in refs read inside the loop to avoid tearing down/rebuilding the whole THREE scene on every media-query flip; only re-run the effect for changes that truly require pass reconfiguration (bloom add/remove).
• Label projection cost: 2D fallback for webglcontextlost must not also keep the (now dead) RAF running — ensure the child fully unmounts.
• No mainnet/testnet/Worker/migration concerns — this is client-only; the only platform risk is the three example-module import paths (three/examples/jsm/...) which already resolve under Vite.
• aria-hidden labels: removing it (line 560) to make labels focusable can flood the a11y tree for dense graphs (up to 40 nodes via MAX_NODES in facts-to-graph.ts); prefer a single visually-hidden listbox of nodes over 40 focusable label divs.

Acceptance-criteria mapping

• Autorotate + bloom + RAF gated on prefers-reduced-motion → Step 2 (RM state/listener) + Step 3 (autoRotate at line 345, RAF at line 362, bloom at line 349, on-demand renderOnce).
• 2D/list fallback from facts-to-graph.ts (+ related-neighbors.ts) when WebGL unavailable → Step 1 (webglSupported) + Step 2 (mode switch) + Step 4 (MemoryGraph2D).
• Esc deselects active node and exits fullscreen → Step 2 (single keydown handler).
• Search results and nodes keyboard-navigable (focus + arrow/enter, aria-activedescendant) → Step 5 (MemorySearchBox combobox/listbox) + Step 4 (focusable 2D rows) + visually-hidden node list for 3D.
• Small viewports (<640px): reduced pixelRatio + bloom, usable graph + detail panel → Step 1 (isCompactViewport) + Step 3 (pixelRatio at line 333, bloom strength) + Step 6 (bottom-sheet @media).
• Switching to fallback disposes renderer/composer (no leaked context) → Step 2 (unmount on !webglOk) + Step 3 (extended cleanup incl. composer.dispose()/bloom.dispose(), webglcontextlost).

Effort
M — ~2.5-3.5 person-days. The component split + extraction is the bulk (~1.5 pd) but mechanical; reduced-motion/on-demand-render and the 2D fallback are ~1 pd; search a11y, Esc, mobile CSS, and tests ~0.5-1 pd. Risk concentrated in the THREE lifecycle (dispose + on-demand render), which the manual GL-context-count check validates.

---

Implementation plan generated for execution (planning phase). Part of the roadmap (#4).