Step 5: Memory — SQLite persistence, embeddings, hybrid retrieval, summarization

[hackertron]

Context

Steps 1-4 are complete (types, config, providers, tools, runtime). The agent can think and act but forgets everything when it stops. Step 5 adds persistent memory.

Scope

SQLite schema + migrations

• Sessions table (id, name, created_at, updated_at, archived)
• Conversation events table (session_id, role, content, tool_calls, created_at)
• Memory chunks table (id, content, source, tags, embedding blob, created_at)
• FTS5 virtual table on chunks
• Session state table (summary, summary_epoch, scratchpad)
• Schema version tracking

Embedding backends

• EmbeddingBackend interface implementation
• OpenAI embeddings (text-embedding-3-small)
• Ollama embeddings (local, free — nomic-embed-text)
• Configured via memory.embedding_backend in config

Hybrid retrieval pipeline

• Vector search via sqlite-vec (cosine similarity)
• FTS5 search (BM25 scoring)
• Normalize scores (min-max within each result set)
• Weighted merge (configurable vector_weight + fts_weight)
• Return top_k results

Rolling summarization

• Detect when context budget is approaching limit (chars/4 estimate already in runtime)
• Summarize unsummarized messages since last epoch
• Call provider with summarization prompt
• Update session state with new summary + epoch
• Replace old messages with summary in context

Memory tools

• memory_search (ReadOnly) — search persistent memory
• memory_save (SideEffecting) — save information to memory

Conversation persistence

• Store conversation events as they happen
• Load conversation history when resuming a session
• Session create/get/list/update/archive via SessionStore interface

Scratchpad

• Per-session key-value scratchpad for agent working notes
• Persisted in session_state table

Interfaces already defined

• internal/types/memory.go — Memory, MemoryRetrieval, EmbeddingBackend
• internal/types/session.go — SessionStore, SessionRecord

Files to create

internal/memory/
├── sqlite.go          # SQLiteMemory implementing MemoryRetrieval
├── schema.go          # DDL + migrations
├── embedding_openai.go
├── embedding_ollama.go
├── hybrid.go          # Hybrid retrieval pipeline
├── summarizer.go      # Rolling summarization
└── memory_test.go

internal/tool/
├── memory_search.go   # memory_search tool
└── memory_save.go     # memory_save tool

Dependencies

• modernc.org/sqlite (pure Go SQLite)
• github.com/nicholasgasior/govec or similar for sqlite-vec

Not included

• Multi-user isolation
• Audit logging
• Cost tracking per memory operation