Skip to content

feat(kb): add knowledge base generation from existing documents#5

Open
bkuberek wants to merge 1 commit into
mainfrom
feat/kb-generation
Open

feat(kb): add knowledge base generation from existing documents#5
bkuberek wants to merge 1 commit into
mainfrom
feat/kb-generation

Conversation

@bkuberek

Copy link
Copy Markdown
Owner

Summary

  • Add mkcv kb generate and mkcv kb update CLI commands that synthesize structured, lossless knowledge bases from existing career documents using an LLM
  • Support multi-format input: PDF, Markdown, DOCX, HTML, TXT — single files, multiple files, or recursive directory scanning with glob filtering
  • Follow hexagonal architecture with new DocumentReaderPort, MultiFormatDocumentReader adapter, KBGenerationService, and Jinja2 prompt templates
  • Chunked processing for large document sets (>100K chars) with iterative merge strategy
  • 113 new tests covering models, exceptions, adapter, service, factory, and CLI (1554 total, all passing)

New Commands

# Generate a KB from documents
mkcv kb generate resume.pdf linkedin-export.html ./career-docs/

# Generate with custom name and output
mkcv kb generate ./docs/ --name "Tech Career" --output tech-career.md

# Update an existing KB with new documents
mkcv kb update new-role.docx --kb career.md

# Filter files in directories
mkcv kb generate ./docs/ --glob "*.pdf"

Architecture

Layer New Files Role
Port core/ports/document_reader.py DocumentReaderPort protocol
Models core/models/document_content.py, kb_generation_result.py Typed data structures
Exceptions core/exceptions/kb_generation.py DocumentReadError, KBGenerationError
Adapter adapters/filesystem/document_reader.py MultiFormatDocumentReader (5 formats)
Service core/services/kb_generation_service.py KBGenerationService with generate/update
Prompts prompts/kb_generate.j2, kb_update.j2 Lossless synthesis templates
CLI cli/commands/kb.py Cyclopts sub-app with generate/update
Config config/settings.toml [default.kb] section

Test plan

  • Unit tests for DocumentContent and KBGenerationResult models
  • Unit tests for DocumentReadError and KBGenerationError exceptions
  • Unit tests for MultiFormatDocumentReader (all 5 formats, directory scanning, glob filtering)
  • Unit tests for KBGenerationService (generate, update, chunked processing, error handling)
  • Integration tests for factory wiring (create_kb_generation_service)
  • CLI tests for mkcv kb generate and mkcv kb update (help, args, mock service)
  • Ruff lint + format clean
  • mypy --strict passes (130 source files)
  • Full test suite: 1554 passed

🤖 Generated with Claude Code

Add `mkcv kb generate` and `mkcv kb update` CLI commands that use an LLM
to synthesize a structured, lossless knowledge base from career documents
(PDF, Markdown, DOCX, HTML, TXT). Supports single files, multiple files,
and recursive directory scanning with glob filtering. Follows hexagonal
architecture with new DocumentReaderPort, MultiFormatDocumentReader adapter,
KBGenerationService, and Jinja2 prompt templates.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant