Skip to content

Align Hermes passive DKG memory recall#389

Open
Jurij89 wants to merge 4 commits intomainfrom
codex/hermes-six-layer-passive-memory
Open

Align Hermes passive DKG memory recall#389
Jurij89 wants to merge 4 commits intomainfrom
codex/hermes-six-layer-passive-memory

Conversation

@Jurij89
Copy link
Copy Markdown
Contributor

@Jurij89 Jurij89 commented May 4, 2026

Source issue

Closes #385

Summary

  • Refreshed PR Align Hermes passive DKG memory recall #389 onto latest main (0af092fb) after PR Codex/issue 396 private policy swm gate #420 merged; current head is f9d67b2888c863202263fcac8482597239ba744a.
  • Replaced Hermes passive prefetch() assertion-only recall with six-layer DKG fan-out behavior aligned with memory_search.
  • Added Node UI selected-project propagation for Hermes' default OpenAI-compatible transport so passive prefetch can see the selected context graph before the model turn.
  • Added provenance-rich passive recall blocks with context graph id, DKG view, OpenClaw-style layer label, source, score, and path.
  • Added scoped dkg_memory.context_graph_id support while keeping unscoped simple memory writes defaulted to agent-context / memory.
  • Tightened memory-search keyword selection, SPARQL ordering, and selected-project result preservation so unique project hits are not crowded out by stale agent-context snippets.
  • Added OpenClaw-style Hermes provider recall logs for hook/tool caller, selected project, layer count, raw hit count, and per-layer hit breakdown.
  • Added provider-side and daemon-side stripping for adapter-generated recalled-memory blocks and Node UI routing markers before Hermes chat persistence/import.

OpenClaw parity notes

  • Hermes passive recall always queries agent-context WM/SWM/VM.
  • Hermes passive recall and explicit memory_search query project WM/SWM/VM only when the calling surface supplies a selected project via context_graph_id / target_context_graph or the Node UI selected-project marker; static context_graph config does not count as current selection.
  • Layer labels match OpenClaw: agent-context-wm, agent-context-swm, agent-context-vm, project-wm, project-swm, project-vm.
  • Trust ranking matches OpenClaw weighting: WM x1.0, SWM x1.15, VM x1.3.
  • Recall results dedupe by context graph plus memory URI/text hash, keeping the higher-trust layer for the same memory.
  • No adapter-to-adapter imports were added; OpenClaw remains the parity reference.

Files changed

File What
packages/adapter-hermes/hermes-plugin/__init__.py Hermes provider/tool schema, six-layer passive recall, Node UI marker parsing/stripping, ranking, persistence safety
packages/adapter-hermes/test/hermes-adapter.test.ts Provider/tool/schema/passive recall/provenance/marker/ranking regression coverage
packages/cli/src/daemon/routes/hermes.ts Node UI selected context marker for Hermes OpenAI-compatible chat requests
packages/cli/test/daemon-hermes.test.ts Daemon route coverage for forwarded selected context markers
packages/adapter-hermes/README.md Hermes behavior/setup docs
docs/setup/SETUP_HERMES.md Hermes setup docs
packages/cli/skills/dkg-node/SKILL.md DKG memory behavior guidance

Behavioral details

  • Passive prefetch uses a shared Hermes-local helper with memory_search for query planning, fan-out, ranking, dedupe, and provenance.
  • agent-context recall always runs across WM/SWM/VM. A selected project graph is included only when supplied by the current calling surface.
  • Node UI Hermes OpenAI-compatible requests append a machine-readable <dkg-node-ui-context ... /> marker to the forwarded user message when a project is selected. The Hermes provider parses that marker for prefetch scope and strips it before chat persistence.
  • Passive injected context is emitted as <recalled-memory data-source="dkg-auto-recall"> with escaped snippet text and untrusted-context framing.
  • dkg_memory.context_graph_id stores scoped notes under separate cache buckets and queues writes with the target graph id. Unscoped writes continue to target agent-context / memory.
  • Present but non-string or blank context_graph_id values on dkg_memory and memory_search fail closed before cache/query/write routing. Optional JSON null is treated as omitted.
  • Existing Hermes profiles that previously stored provider memory in a custom configured context_graph are read only as a legacy fallback after a successful empty agent-context provider-memory read. Failure-shaped default reads do not widen recall to the legacy graph.
  • Memory search now drops generic instruction words, scores row matches inside SPARQL, orders by score before raw limiting, and then applies adapter-side trust/source ranking.
  • memory_search no longer filters out short literals with a hard STRLEN >= 20 cutoff.
  • memory_search dedupe keys include context graph, URI, predicate, and literal content so multiple dkg_memory notes under one assertion subject are preserved.
  • Adapter-generated recalled-memory openings without a closing tag are stripped through EOF before persistence; unrelated malformed tags are preserved.

Test evidence

Final local validation after updating onto main at 0af092fb:

  • python -m py_compile packages/adapter-hermes/hermes-plugin/__init__.py packages/adapter-hermes/hermes-plugin/client.py passed.
  • git diff --check passed.
  • pnpm --filter @origintrail-official/dkg-adapter-hermes test -- hermes-adapter.test.ts passed: 87 tests. This includes Node UI context marker parsing/stripping, unique-token SPARQL ranking coverage, selected-project crowding regression coverage, and OpenClaw-style hook recall log coverage.
  • pnpm --filter @origintrail-official/dkg test -- test/daemon-hermes.test.ts --reporter=verbose passed: 66 tests. The command still prints the known post-run Windows message, The system cannot find the path specified., after exiting successfully.
  • pnpm --filter @origintrail-official/dkg build passed.

Earlier parity/rebase evidence retained from prior rounds:

  • pnpm --filter @origintrail-official/dkg-adapter-openclaw test -- dkg-client.test.ts passed: 58 tests after the prior main refresh touched that OpenClaw test path.
  • pnpm --filter @origintrail-official/dkg-core build passed after pulling main to refresh workspace exports.
  • pnpm --filter @origintrail-official/dkg-adapter-openclaw test -- before-prompt-build-hook.test.ts memory-integration.test.ts memory-search-tool.test.ts ChatTurnWriter.test.ts ChatTurnWriter.sanitize.test.ts plugin.test.ts dkg-client.test.ts passed: 7 files, 451 tests.
  • pnpm --filter @origintrail-official/dkg-agent... build passed.
  • pnpm --filter @origintrail-official/dkg-epcis build passed.

Manual verification status

  • No live Hermes gateway or Node UI E2E was run in this workspace.
  • Manual user testing on previous pushed builds surfaced three issues now addressed: selected Node UI context was not visible to upstream Hermes passive prefetch(), generic query words could crowd out unique IDs before adapter ranking, and selected-project WM hits could still be omitted from the final top-5 injected snippets when stale agent-context matches were abundant.
  • Targeted Python provider, daemon route, dependency-build, CLI build, and selected-project crowding regression coverage passed locally after the fixes.

Security / persistence notes

  • Recalled-memory blocks are treated as read-only prompt context and are stripped before provider sync_turn() persistence.
  • Node UI context routing markers are parsed only for passive recall scope and stripped before provider chat persistence.
  • /api/hermes-channel/persist-turn strips adapter-generated recalled-memory blocks before chat storage, transition recording, and extraction import.
  • Strip tests cover mixed-case sentinel tags, single-quoted and unquoted data-source attributes, malformed unmatched adapter-generated openings, and Node UI context markers so recalled/routing context does not boomerang or cause data loss.
  • Scoped dkg_memory.context_graph_id avoids cache and queue bleed between default personal memory and project notes.
  • Legacy custom-graph provider memory fallback is read-only, fallback-only, and disabled on failed default reads; new unscoped dkg_memory writes still use agent-context unless context_graph_id is explicit.

Known risks or follow-ups

  • No broad refactor into a neutral cross-adapter recall module was attempted in this focused PR.
  • GitHub CI was restarted by the final push and may still be running while manual testing begins.

Comment thread packages/adapter-hermes/hermes-plugin/__init__.py
Comment thread packages/adapter-hermes/hermes-plugin/__init__.py
Comment thread packages/adapter-hermes/hermes-plugin/__init__.py Outdated
@Jurij89 Jurij89 force-pushed the codex/hermes-six-layer-passive-memory branch from ba083f7 to 5f20c49 Compare May 5, 2026 17:52
Comment thread packages/adapter-hermes/hermes-plugin/__init__.py
Comment thread packages/adapter-hermes/hermes-plugin/__init__.py Outdated
@Jurij89 Jurij89 force-pushed the codex/hermes-six-layer-passive-memory branch from 5f20c49 to 35975da Compare May 5, 2026 18:00
Comment thread packages/adapter-hermes/hermes-plugin/__init__.py Outdated
@Jurij89 Jurij89 force-pushed the codex/hermes-six-layer-passive-memory branch from 35975da to 504ddc0 Compare May 5, 2026 18:06
Comment thread packages/adapter-hermes/hermes-plugin/__init__.py Outdated
Comment thread packages/adapter-hermes/hermes-plugin/__init__.py Outdated
@Jurij89 Jurij89 force-pushed the codex/hermes-six-layer-passive-memory branch from 504ddc0 to a5c18cb Compare May 6, 2026 11:29
Comment thread packages/cli/src/daemon/routes/hermes.ts Outdated
Comment thread packages/adapter-hermes/hermes-plugin/__init__.py Outdated
Comment thread packages/adapter-hermes/hermes-plugin/__init__.py Outdated
@Jurij89 Jurij89 force-pushed the codex/hermes-six-layer-passive-memory branch from a5c18cb to af1aeca Compare May 6, 2026 11:35
Comment thread packages/adapter-hermes/hermes-plugin/__init__.py Outdated
Comment thread packages/adapter-hermes/hermes-plugin/__init__.py Outdated
@Jurij89 Jurij89 force-pushed the codex/hermes-six-layer-passive-memory branch from af1aeca to e515e53 Compare May 6, 2026 11:42
Comment thread packages/adapter-hermes/hermes-plugin/__init__.py
Comment thread packages/adapter-hermes/hermes-plugin/__init__.py
Comment thread packages/adapter-hermes/hermes-plugin/__init__.py Outdated
@Jurij89 Jurij89 force-pushed the codex/hermes-six-layer-passive-memory branch from e515e53 to 260ec59 Compare May 6, 2026 11:52
Comment thread packages/adapter-hermes/hermes-plugin/__init__.py
Comment thread packages/cli/src/daemon/routes/hermes.ts
@Jurij89 Jurij89 force-pushed the codex/hermes-six-layer-passive-memory branch from 260ec59 to 6ac9053 Compare May 6, 2026 12:00
Comment thread packages/adapter-hermes/hermes-plugin/__init__.py Outdated
Comment thread packages/adapter-hermes/hermes-plugin/__init__.py
@Jurij89 Jurij89 force-pushed the codex/hermes-six-layer-passive-memory branch from 6ac9053 to 8194452 Compare May 6, 2026 12:06
Comment thread packages/adapter-hermes/hermes-plugin/__init__.py Outdated
Comment thread packages/adapter-hermes/hermes-plugin/__init__.py
@Jurij89 Jurij89 force-pushed the codex/hermes-six-layer-passive-memory branch from 8194452 to 8b12cf9 Compare May 6, 2026 13:09
@Jurij89 Jurij89 force-pushed the codex/hermes-six-layer-passive-memory branch from 8b12cf9 to 343952b Compare May 6, 2026 15:30
@Jurij89 Jurij89 force-pushed the codex/hermes-six-layer-passive-memory branch from 343952b to 014dfb0 Compare May 6, 2026 15:39
Jurij Skornik added 2 commits May 6, 2026 18:37
…dkg_memory in system prompt

Two fixes for the remaining issues blocking PR #389 validation.

Issue B (contextGraphId not reaching Hermes for some turns):

- The Node UI body builder used a ternary that omitted contextGraphId
  entirely when activeProjectId was null/undefined. On chat-reset / state-
  race paths where activeProjectId briefly went null while the visible
  badge still indicated a project, follow-up turns shipped without the
  field; the daemon's buildHermesOpenAiUserMessage early-returned plain
  text; the <dkg-node-ui-context> marker never made it to Hermes; passive
  recall silently scoped to agent-context-only.

- buildLocalAgentChatBody now always includes contextGraphId (as null
  when not selected) so the daemon can distinguish "no project" from
  "field accidentally omitted". optionalTrimmedString already resolves
  null to undefined downstream, so no daemon parser changes needed.

- routes/hermes.ts adds a bounded per-session contextGraphId cache
  (200-entry LRU). When a turn arrives with no contextGraphId but has a
  sessionId that previously carried one, the cache restores the value
  before payload forwarding. This fixes the symptom regardless of which
  UI path drops the field — Hermes' passive recall now stays scoped to
  the operator's selected project across turns.

- New test in daemon-hermes.test.ts exercises the cache fallback across
  five send calls (prime → recover-from-cache → isolated-session →
  switch-projects → recover-from-cache-after-switch).

Issue A mitigation (dkg_memory.context_graph_id schema visibility):

- Adds one targeted line to the Hermes Node UI system prompt when a
  contextGraphId is present: "When persisting project-scoped notes for
  this session, call dkg_memory with context_graph_id: '<id>'."

- This is a live, in-prompt signal that competes with stale
  <recalled-memory> snippets describing the older 4-field dkg_memory
  shape. Doesn't fix the underlying recall-pollution but gives the LLM
  a fresh reason to pick up the new field each turn.

- New test asserts the system prompt contains "dkg_memory" + the
  context_graph_id name + the project id when contextGraphId is supplied.

Tests:
- 68 daemon-hermes tests pass (was 66, +2 new — cache fallback +
  dkg_memory nudge).
- 87 hermes-adapter tests still pass.
- Node UI tsc --noEmit clean.

Out of scope (separate follow-ups if confirmed):
- The React-state root cause that lets activeProjectId go null while the
  UI shows a project selected.
- A recalled-memory filter that excludes stale tool-introspection text.
- Equivalent per-session cache for OpenClaw's uiContextGraphId
  (OpenClaw has no passive recall so the symptom is less acute, but the
  same pattern would apply for parity).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Align Hermes passive memory recall with OpenClaw six-layer DKG memory behavior

1 participant