[FEATURE] Support long-term memory for agents

[supreme-gg-gg]

📋 Prerequisites

• I have searched the existing issues to avoid creating a duplicate
• By submitting this issue, you agree to follow our Code of Conduct

📝 Feature Summary

Add support for long-term memory

❓ Problem Statement / Motivation

Why do we need long term memory?

Sessions can track history (events) and temporary data for a single ongoing conversation. But in order for agents to recall information from past conversations (from the same or different session), we need a memory service to support "long-term knowledge", similar to a searchable archive or knowledge library that the agent can consult, potentially containing information from many past chats or other sources.

💡 Proposed Solution

System Overview

• Agent (Python ADK): Uses KagentMemoryService to send/retrieve memories via HTTP and generate embeddings, configured from the CRDs
• Controller (Go): Exposes /api/memory endpoints to manage vector storage
• Storage (Postgres): Stores embeddings and metadata in the agent_memory table.

Components

1. HTTP server endpoints (Go)

• POST /api/memory/sessions: Accepts (text, vector) pairs. Stores in agent_memory.
• POST /api/memory/search: Accepts (query_vector). Performs cosine similarity search in pgvector

2. Kagent ADK (Python)

KagentMemoryService will BaseMemoryService with two methods:

• add_session_to_memory: Uses LiteLLM to generate embeddings using the configured embedding model, send vectors to Kagent API
• search_memory: Embed query and send query_vector to Kagent API

This will be created and passed to the Runner if memory is enabled on the agent. The agent will use tools to either optionally or preload memory. After each agent execution, a callback will save the session to memory using the memory service. These are documented further in the ADK memory interface: https://google.github.io/adk-docs/sessions/memory/

3. Embedding Model Config

apiVersion: kagent.dev/v1alpha2
kind: ModelConfig
spec:
  embedding:
    provider: OpenAI # or AzureOpenAI, Vertex, Ollama
    model: "text-embedding-3-small"

4. Agent Configuration

spec:
  declarative:
    memory:
      enabled: true
      modelConfig: "embedding-model-config"

5. Data Model

The user can configure whether to extend the database to use vector-extensions in Helm Chart:

// helm/kagent/values.yaml
database:
  postgres:
    vector:
      enabled: true # Default: true

We will use pgvector for Postgres (production) and sqlite-vector (https://github.com/sqliteai/sqlite-vector) for sqlite (development).

Table: agent_memory

id: UUID
agent_name: String (Tenant)
user_id: String (Tenant)
content: Text (Chunk)
embedding: Vector (Size varies by model)
metadata: JSONB (Source session ID, timestamp)

🔄 Alternatives Considered

• Use the Qdrant MCP server or similar options: https://google.github.io/adk-docs/tools/third-party/qdrant/ However this requires using a cloud-hosted solution for the vector DB. Note that if you wish to use an external Vector database (like qdrant, chroma, pinecone), you can still use the MCP approach and disable the internal pgvector memory service.

• Use pinecone or qdrant in cluster: this was a popular approach perhaps a year ago, but right now the best practice is to keep relational data and vector data in the same database as extensions like pgvector mature and reach similar performance as dedicated vector databases. This not only provides tighter integration and less overhead, but also better metadata query performance.

🎯 Affected Service(s)

Multiple services / System-wide

📚 Additional Context

The feature will first be rolled out in ADK. Then, support to BYO agents will be added via the kagent packages to LangGraph, CrewAI, OpenAI SDK.

🙋 Are you willing to contribute?

• I am willing to submit a PR for this feature

[EItanya]

This design looks great overall, I just have a few comments:

I think this addition to the ModelConfig may get confusing. Currently there's no nesting in that object, so I think we either need to re-use it as is, or come up with a new plan.

apiVersion: kagent.dev/v1alpha2
kind: ModelConfig
spec:
  embedding:
    provider: OpenAI # or AzureOpenAI, Vertex, Ollama
    model: "text-embedding-3-small"

Why do we need enabled: true I think that the field being set at all can indicate that it's enabled?

4. Agent Configuration

spec:
declarative:
memory:
enabled: true
modelConfig: "embedding-model-config"

Is there any downside to just adding this extension by default and saving users the confusion?

The user can configure whether to extend the database to use vector-extensions in Helm Chart:

// helm/kagent/values.yaml
database:
  postgres:
    vector:
      enabled: true # Default: true

[supreme-gg-gg]

I agree that we can just reuse ModelConfig as it is, since when used in memory.modelConfig in agent spec it will just be recognised as an embedding model.

There shouldn't be much downside to always enabling pgvector extension, I was thinking that some users might not want to use memory and prefer not to install unused extensions to their database. But the extension is quite lightweight so I agree for now we can just always include it.

Also tagging @lets-call-n-walk since you asked about memory in discord

[dimetron]

Hi @supreme-gg-gg before we introduce pgvec and sqlite_vec hard dependency
Can we start with memory as external service first ?

I think Memory requiremnents might quickly overgrow simple implementation
and we should keep it pluggable and integrated via API/hooks etc.

We should keep memory as external service allowing alternative implementations.
See: https://qdrant.tech/documentation/fastembed/fastembed-rerankers/
Most simple vector search, could be extended with re-ranking etc

It could be as well memory with Knowledge Graph relations and not depend on vector / semantic search https://github.com/modelcontextprotocol/go-sdk/blob/main/examples/server/memory/main.go
with simple DB table.

With slight rework of Qdrant MCP https://github.com/qdrant/mcp-server-qdrant/tree/master we can create kagent-memory RAG service with required API's

• add_session_to_memory
• search_memory

Embeddings:
There are too many providers to support https://qdrant.tech/documentation/embeddings/
https://www.firecrawl.dev/blog/best-vector-databases-2025
We should be able to provide simple default embedding service which works everywhere with zero effort on CPU. Popular examples: all-MiniLM-L6-v2 or Gemma https://github.com/EricLBuehler/mistral.rs/blob/master/mistralrs/examples/embedding_gemma/main.rs

Considerations:

• Important to pay attention to dimensional dense 384 or 768 which impact pgvec performance.
• Our native SQLLite drivers may support vectors already see https://gitlab.com/cznic/sqlite#examples
• How we will delete memory ?

PS:
There are several projects with focus only on memory with good examples of API and integration with popular frameworks. It might be useful to review them before reinventing the thing.

https://memorilabs.ai/docs/
https://github.com/bobmatnyc/kuzu-memory
https://github.com/getzep/graphiti
https://github.com/topoteretes/cognee
https://github.com/CaviraOSS/OpenMemory

[supreme-gg-gg]

I think Memory requiremnents might quickly overgrow simple implementation and we should keep it pluggable and integrated via API/hooks etc.

I agree that memory requirements can outgrow simple implementations. However, I think this design provides more of a "batteries-included" experience. Users who need specialized stores like Qdrant or Knowledge Graphs can already use the MCP approach you mentioned to swap out the memory layer (and disable the built-in pgvector memory).

Since MCP is already our standard for external tools, it acts as the "API/hook" you’re looking for without us having to build custom integrations for every DB provider, which would be difficult since each vector db provider uses a different API / client library.

With slight rework of Qdrant MCP https://github.com/qdrant/mcp-server-qdrant/tree/master we can create kagent-memory RAG service with required API's

• add_session_to_memory
• search_memory

I did consider Qdrant MCP at first since ADK has a built-in support. But I looked at some benchmarks and pgvectors with pgvectorscale actually can outperform Qdrant in throughput in some cases, and I think the firecrawl article you mentioned has similar results. Plus, having to manage a separate vector db will introduce a lot more ops overhead and pg extension keeps the data more centralised.

But I was looking at some numbers and realised that vectors can take up a huge proportion of db space if the memory store is heavily used and cause bloat in postgres, since they use much more storage (and their index) than regular data. This might be an argument for having a separate vector store.

Another thing that's valid is that vector-based memory store might not suffice in the long-run, we might prefer something like a hybrid between vector embeddings + graph, as how Mem0 does it (but their implementation requires a graph database which is kinda an overkill for now), or something along those lines. With that being said I think to get things started a simple vector memory would be sufficient for most use cases, and we can always improve it iteratively.

Embeddings: There are too many providers to support https://qdrant.tech/documentation/embeddings/ https://www.firecrawl.dev/blog/best-vector-databases-2025 We should be able to provide simple default embedding service which works everywhere with zero effort on CPU. Popular examples: all-MiniLM-L6-v2 or Gemma https://github.com/EricLBuehler/mistral.rs/blob/master/mistralrs/examples/embedding_gemma/main.rs

For most embedding providers, using LiteLLM should work. I think providing simple default embedding service might be separate from this PR, since right now you can just run it with Ollama and specify that in the ModelConfig (according to the design above) like how you would do with a local LLM.

How we will delete memory ?

Haven't given this one too much thought, for start maybe user request (like Gemini app), given the agent a tool to clear certain memory, or TTL? I will try to come up with a more dynamic method that's more performant, might need some benchmark to measure how well it does though before I can start optimising.

[danehans]

Any consideration of following the ReasoningBank memory framework: https://arxiv.org/pdf/2509.25140

cc: @EItanya

[nujragan93]

I understand that agent chats and conversations can be automatically stored as long-term memory.

1. I’m curious how we can also incorporate external knowledge base articles or connect external services so agents can leverage that information contextually.
2. Should all conversations be stored, or only explicitly chosen ones?

[supreme-gg-gg]

1. I’m curious how we can also incorporate external knowledge base articles or connect external services so agents can leverage that information contextually.

This design includes a built-in vector storage for kagent, which can definitely be used to store "knowledge base" chunks for RAG purposes (in a different or the same table). However, I would consider it as a separate feature from memory because of the way they're used. Memory is often retrieved and extracted and saved automatically while knowledge base should probably be exposed as a tool (via MCP) so the agent can proactively search them only when needed. Plus importing and managing documents in the knowledge base would be separate functionalities.

EDIT: For more info this page on aws AgentCore explains it well: https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/memory-ltm-rag.html

2. Should all conversations be stored, or only explicitly chosen ones?

I’m considering a hybrid approach: using a save_memory tool for explicit user requests / when the agent thinks it's important, and an automated background job that distills key facts after a session ends (or every 10 turns). After extracting content from the session it will embed and store it into memory.

I’d love to hear what everyone else thinks about the 'when' for storage and deletion. There doesn't seem to be any "best-practice" yet and ADK doesn't provide much advice here as well.

[dimetron]

Thanks @supreme-gg-gg - batteries included is a good idea. But in case of PGVec some additional operational overhead is required as well. See instructions for CNPG https://www.tigerdata.com/blog/deploying-timescaledb-vector-search-cloudnativepg-kubernetes-operator

• right now it's possible to run kAgent with Postgres without dependency on extensions
• if we will create such a dependency, every database will need additional extensions setup

Just saying that bundling PGVec and SQLVec might be not the easiest solution. And if we decide to go this way then maybe we should think more how we will manage our custom image for Postgres etc.