Skip to content

feat(agents): add skills system for dynamic knowledge loading#4155

Open
balegas wants to merge 5 commits intomainfrom
balegas/entity-skills-registry
Open

feat(agents): add skills system for dynamic knowledge loading#4155
balegas wants to merge 5 commits intomainfrom
balegas/entity-skills-registry

Conversation

@balegas
Copy link
Copy Markdown
Contributor

@balegas balegas commented Apr 23, 2026

Summary

Adds a skills system to the @electric-ax/agents package so Horton can load specialized knowledge, instructions, and tutorials on demand via use_skill/remove_skill tools. Includes a built-in interactive tutorial that walks users through building a perspectives analyzer with the manager-worker pattern.

Ported from electric-sql/durable-streams feat/entity-skills-registry branch with full rebrand from Darix → Electric Agents naming.

Reviewer Guidance

Approach

The skills system has five layers:

  1. Preamble parser (src/skills/preamble.ts) — extracts YAML frontmatter from .md skill files (description, keywords, arguments, etc.)
  2. Metadata extractor (src/skills/extract-meta.ts) — tries preamble first, falls back to Claude Haiku for incomplete metadata
  3. Registry (src/skills/registry.ts) — scans two directories (built-in + app-provided), caches by content hash, renders catalog with progressive truncation (full → compact → names-only) to fit context budgets
  4. Tools (src/skills/tools.ts) — use_skill injects content into agent context with argument substitution and auto-loads .md reference files; remove_skill cleans up
  5. Horton integration — skills catalog added as a context source, skill tools included when registry has entries, system prompt updated with skills guidance

Key Invariants

  • App skills override base skills with the same name (two-directory model)
  • Content hash caching avoids re-extracting metadata for unchanged files
  • Skill content is returned in the tool result (high attention weight) AND stored in context (persistence across wakes)
  • createBuiltinAgentHandler is now async (was sync) — callers must await it

Non-goals

  • No skill authoring UI — skills are .md files on disk
  • No runtime skill hot-reload — requires server restart
  • No skill dependencies or composition

Trade-offs

Decision Why
Preamble-first, LLM-fallback Fast for well-structured skills, graceful for legacy markdown
Full content in tool result LLM attention weight is highest on recent tool results vs. context sources
Two-directory model Built-in skills ship with the package, app skills are user-customizable without forking

Verification

cd packages/agents
pnpm vitest run  # 37 tests, 8 files — all pass

New test files cover: preamble parsing (8 tests), registry scanning/caching/rendering (11 tests), tool execution/argument substitution (9 tests), end-to-end lifecycle (1 test).

Also includes docker-compose.full.yml fix: postgres:18-alpinepostgres:17-alpine (PG 18 broke volume mount path convention).

Files Changed

File Change
packages/agents/src/skills/types.ts SkillMeta and SkillsRegistry interfaces
packages/agents/src/skills/preamble.ts YAML frontmatter parser
packages/agents/src/skills/extract-meta.ts Metadata extraction with LLM fallback
packages/agents/src/skills/registry.ts Registry: scan, cache, render catalog
packages/agents/src/skills/tools.ts use_skill / remove_skill agent tools
packages/agents/skills/tutorial.md Interactive perspectives analyzer tutorial
packages/agents/skills/tutorial/scaffold/ Starter project files for tutorial
packages/agents/src/agents/horton.ts Skills tools + catalog integration
packages/agents/src/bootstrap.ts Skills registry init at startup (now async)
packages/agents/src/server.ts await async bootstrap + Awaited<> type fix
packages/agents/test/skills-*.test.ts 29 new tests across 4 files
packages/electric-ax/docker-compose.full.yml Pin postgres to 17-alpine

🤖 Generated with Claude Code

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 23, 2026

Codecov Report

❌ Patch coverage is 73.45133% with 90 lines in your changes missing coverage. Please review.
✅ Project coverage is 68.28%. Comparing base (5bf1f48) to head (84052c5).
⚠️ Report is 1 commits behind head on main.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
packages/agents/src/skills/extract-meta.ts 38.23% 21 Missing ⚠️
packages/agents/src/agents/horton.ts 0.00% 20 Missing ⚠️
packages/agents/src/skills/tools.ts 81.48% 20 Missing ⚠️
packages/agents/src/bootstrap.ts 0.00% 11 Missing ⚠️
packages/agents/src/skills/registry.ts 87.91% 11 Missing ⚠️
packages/agents/src/skills/preamble.ts 93.15% 5 Missing ⚠️
packages/agents/src/server.ts 0.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4155      +/-   ##
==========================================
+ Coverage   68.08%   68.28%   +0.19%     
==========================================
  Files         130      134       +4     
  Lines       16211    16543     +332     
  Branches     3907     4019     +112     
==========================================
+ Hits        11037    11296     +259     
- Misses       5170     5243      +73     
  Partials        4        4              
Flag Coverage Δ
packages/agents 48.67% <73.45%> (+10.27%) ⬆️
packages/agents-runtime 81.34% <ø> (ø)
packages/agents-server 65.55% <ø> (ø)
packages/agents-server-ui 0.00% <ø> (ø)
packages/electric-ax 30.46% <ø> (ø)
packages/experimental 87.73% <ø> (ø)
packages/react-hooks 86.48% <ø> (ø)
packages/start 82.83% <ø> (ø)
packages/typescript-client 94.30% <ø> (ø)
packages/y-electric 56.05% <ø> (ø)
typescript 68.28% <73.45%> (+0.19%) ⬆️
unit-tests 68.28% <73.45%> (+0.19%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

balegas and others added 3 commits April 23, 2026 18:57
Adds a skills registry that scans markdown files at startup and provides
use_skill/remove_skill tools to Horton for on-demand knowledge injection.
Includes an interactive tutorial skill for the perspectives analyzer pattern.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@balegas balegas force-pushed the balegas/entity-skills-registry branch from cb0de9f to f7144b3 Compare April 23, 2026 17:58
balegas and others added 2 commits April 23, 2026 19:32
The tutorial skill was over-trimmed and lost important introductory
content about Electric Agents concepts (entities, handlers, wakes,
agent loop, spawning, workers, state). Restore these so new users
get proper context before building their first entity.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The agents package has internal skills (not intent skills), so
intent validate fails with "No SKILL.md files found".

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant