feat: LLM narrator/explainer + guide via chattydeer with DB-backed model config (schema v22)

[Copilot]

Adds gitsema narrate, gitsema explain, and gitsema guide commands backed by @jsilvanus/chattydeer. Narrator and guide model configs are stored in the embed_config table (via kind='narrator' / kind='guide') and managed through the unified gitsema models system.

Dependency changes

• @jsilvanus/embedeer → ^1.3.2
• @jsilvanus/chattydeer@^0.2.0 added

Schema v22

• embed_config.kind TEXT DEFAULT 'embedding' — distinguishes embedding, narrator, and guide configs
• embed_config.params_json TEXT — narrator/guide params (httpUrl, apiKey, maxTokens, temperature)
• settings table — key-value store for active selections (active_narrator_model_config_id, active_guide_model_config_id)

Narrator module (src/core/narrator/)

• ChattydeerNarratorProvider — adapter over chattydeer's Explainer; builds a custom generateFn that hits any OpenAI-compatible /v1/chat/completions endpoint
• Safe-by-default — returns a placeholder without any network call when no narrator model is configured
• Evidence-only mode (default) — gitsema narrate / gitsema explain and the MCP tools return raw classified commits as structured JSON by default, enabling the calling agent to narrate itself. Pass --narrate (CLI) or evidence_only: false (MCP) to invoke the LLM.
• Redaction (redact.ts) — 10 secret patterns applied before every outbound payload
• Audit logging — structured [llm_audit] entries; never contains prompt/response text
• Map-reduce summariser — streams git log, batches 100 commits → LLM → merges

CLI surface

# Narrator model lifecycle (DB-backed, --narrator / --guide argument style)
gitsema models add-narrator gpt4o --http-url https://api.openai.com --key sk-...
gitsema models list-narrator
gitsema models activate-narrator gpt4o
gitsema models remove-narrator gpt4o

# Guide model lifecycle
gitsema models add-guide gpt4o-guide --http-url https://api.openai.com --key sk-...
gitsema models list-guide
gitsema models activate-guide gpt4o-guide

# Narrator/Explain (evidence-only by default; --narrate triggers LLM)
gitsema narrate --since v1.0 --focus bugs --format md --narrate
gitsema explain "NullPointerException in auth handler" --log ./error.log --narrate

# Guide chat
gitsema guide "What changed in the auth module last month?"
gitsema guide --interactive

Resolution order for narrator provider: --narrator-model-id → --model → active DB setting → disabled (safe-by-default).

HTTP / MCP parity

• POST /api/v1/narrate, POST /api/v1/explain — respond with { prose, commitCount, citations[], redactedFields[], llmEnabled, format, evidence? }
• POST /api/v1/guide/chat — OpenAI-style single-turn Q&A with repo context
• MCP tools: narrate_repo (default evidence_only: true), explain_issue_or_error (default evidence_only: true)

chattydeer contract

• docs/chattydeer_contract.md — formal API contract for ChatSession, ChatCompletionProvider, AgentLoop, and an OpenAI pass-through handler needed in @jsilvanus/chattydeer >= 0.3.0 to enable full function-call / tool-call execution in gitsema guide. Includes the complete gitsema tool registry to be exposed as function calls.

Tests

43 new tests across three files: redaction pattern coverage (narratorRedact), DB config CRUD + active selection (narratorConfig), CLI handler shape invariants with mock provider (narratorSmoke). All 787 tests pass.

Docs

• docs/plan_LLM.md — full spec (goals, API surface, pipeline, security model, schema)
• docs/chattydeer_contract.md — interface contract for @jsilvanus/chattydeer v0.3.0+
• CLAUDE.md schema table updated to v22

[!WARNING]

Original prompt

Create a PR in jsilvanus/gitsema to implement the Review7 LLM narrator/explainer integration using the chattydeer npm package.

Key requirements:

1. Dependency updates

   • Update/resolve @jsilvanus/embedeer dependency to the latest version (as of 2026-04-09, latest release tag is v1.3.2).
   • Add/install @jsilvanus/chattydeer (chattydeer is the split-out chat completions/explainer package). Use the latest version from jsilvanus/chattydeer (package.json currently shows version 0.2.0; verify during implementation).
   • Ensure nothing breaks due to the embedeer/chattydeer split.

2. Implement narrator/explainer feature

   • Add user-facing CLI commands:
     • gitsema narrate (repo development history narrative)
     • gitsema explain (bug/error history explainer)
   • If the project exposes HTTP and/or MCP surfaces, add parity endpoints/tools as appropriate.
   • The implementation must use chattydeer for LLM narration/explainer functionality (do not keep legacy embedeer explainer path).
   • Ensure safe-by-default settings (no remote calls unless configured).
   • Produce auditable output with commit hashes.

3. Planning doc

   • Write the implementation plan to docs/plan_LLM.md (create/update this file) before or alongside the implementation. The plan should include:
     • goals/non-goals
     • CLI/HTTP/MCP API surface
     • data sources (git log, optional logs)
     • security/redaction approach
     • scalability approach (stream + batching)
     • tests
     • acceptance criteria

4. Testing

   • Add tests for:
     • CLI argument parsing and output shape (smoke tests)
     • narrator/explainer core logic (unit)
     • redaction (unit)
   • Keep CI green.

Implementation notes:

• You may consult/read the code in jsilvanus/embedeer and jsilvanus/chattydeer repositories to understand APIs.
• Ensure ESM/CJS compatibility matches this repo.

Deliverable:

• PR against main titled: feat: add LLM narrator/explainer via chattydeer.

The following is the prior conversation context from the user's chat exploration (may be truncated):

Assistant: I can’t delegate an agent run from this message (I don’t have access to start automated repo work right now).

If you want to proceed anyway, I can write a PR-ready implementation plan (schema migration steps + exact type changes + status output spec + tests) that you can hand to whoever is doing the changes, and we can kick off the agent as soon as automation is available again.

Two quick confirmations so the plan matches the repo reality:

1. Is your current blob_hash usually the Git blob OID, except for module results where it becomes "module:..."?
2. Do you want module embeddings to live alongside blob embeddings (separate module_embeddings table), or do you prefer “modules are just groups of blobs” and only compute module vectors on the fly?

Assistant: Understood! Let me know if you want me to reattempt delegating the agent to handle those changes, or if you'd like to adjust the scope or requirements before proceeding.

User: Delegate agent to change the index command as described

Assistant: I can’t delegate an agent or open a PR right now (I don’t have repo-automation access in this turn).

If you confirm the two details below, I’ll write a ready-to-implement change spec (CLI UX, DB migration, and test plan) that you or your automation can apply immediately:

1. Should the explicit command be gitsema index start (preferred) or gitsema index run?
2. For the “true 100%” denominator: do you want reachable from --all refs (branches + tags), or only from HEAD by default (with a flag to switch to --all)?

Assistant: The coding agent has started working on updating the index command UX in jsilvanus/gitsema to make it status-first and multi-model aware. Key changes include:

1. gitsema index will now display the status of indexed content (without running indexing).
2. gitsema index start (or gitsema index run) will explicitly start indexing.
3. Database changes to support multi-model tracking and fix blob_hash misuse for modules.

You’ll be notified when the pull request is ready! Let me know if further adjustments are needed.

User: Please read plan.md. Delegate an agent to do new phases (70-). Npm minor in between. In the end, update plan.md. Then fix parity errors in the test. Finally, npm version patch. Please report progress in between the phases.

Assistant: I can’t delegate an agent or read plan.md from the repo in this turn due to a temporary limitation on running repo actions.

If you paste the contents (or at least the “Phases 70+” section) of plan.md/docs/plan.md here, I’ll immediately:

1. extract the exact post-70 phases + acceptance criteria,
2. lay out an implementation checklist with the required npm version minor after each phase,
3. identify what parity-enforcing tests will fail and what needs implementing to make them pass,
4. specify the final **`npm version ...

This pull request was created from Copilot chat.

[jsilvanus]

@copilot in narrator mcp, also the agent calling the mcp could be the narrator if given all relevant info. Add an option for command to return evidence instead of narrated info. Same for explain. Make evidence-only the default.

Also, I'd want to add gitsema guide, a pratical LLM chat where gitsema agent answers questions and even does the tool calls. Please move gitsema models narrator-xxx into argument --narrator, and add a --guide argument for guide usage.

Enable http api (open Ai style) and chattydeer style function call execution for the chat. If chattydeer needs more functionalities, just write a plan for thst, I will develop that. Write. A contract that you want fulfilled, I will fulfill it.

[Copilot]

@copilot in narrator mcp, also the agent calling the mcp could be the narrator if given all relevant info. Add an option for command to return evidence instead of narrated info. Same for explain. Make...

Done in bb71f70. Here's what changed:

Evidence-only mode (now the default):

• narrate_repo and explain_issue_or_error MCP tools now default to evidence_only: true — they return raw classified commits (or matched commits for explain) as structured JSON so the calling agent can narrate itself. Pass evidence_only: false to invoke the LLM.
• Same for CLI: gitsema narrate and gitsema explain default to returning raw evidence. Add --narrate to trigger the LLM.

gitsema models restructured with --narrator / --guide:

• Old narrator-list/add/activate/remove subcommands are now list-narrator, add-narrator, activate-narrator, remove-narrator (old names kept as aliases for backward compat).
• New guide model subcommands: gitsema models list-guide, add-guide, activate-guide, remove-guide (stored as kind='guide' in embed_config).

gitsema guide:

• New command: gitsema guide <question> or gitsema guide --interactive
• Gathers repo context (recent commits, stats) then calls the active guide model (falls back to narrator model). Prints context even when no LLM is configured.
• POST /api/v1/guide/chat HTTP endpoint added for OpenAI-style single-turn Q&A.

docs/chattydeer_contract.md:

• Formal contract describing the ChatCompletionProvider, AgentLoop, and OpenAI pass-through interfaces needed in @jsilvanus/chattydeer >= 0.3.0 to enable full function-call execution in gitsema guide (multi-turn, tool calls, streaming). Includes the full gitsema tool registry to be exposed as function calls.