ReflexioAI · yyiilluu · Jun 29, 2026
diff --git a/reflexio/server/README.md b/reflexio/server/README.md
@@ -72,7 +72,7 @@ Description: FastAPI backend server that processes user interactions to generate
 - **Identity/config**: `GET /api/whoami`, `GET /api/my_config`, `GET /api/get_config`, `POST /api/set_config`, `POST /api/update_config`
 - **Publish/direct writes**: `POST /api/publish_interaction`, `POST /api/add_user_profile`, `POST /api/add_user_playbook`, `POST /api/add_agent_playbook`
 - **Retrieval**: `POST /api/get_requests`, `POST /api/get_interactions`, `GET /api/get_all_interactions`, `POST /api/get_profiles`, `GET /api/get_all_profiles`, `POST /api/get_user_playbooks`, `POST /api/get_agent_playbooks`, `POST /api/get_agent_success_evaluation_results`
-- **Search/stats**: `POST /api/search`, `POST /api/search_profiles`, `POST /api/rerank_user_profiles`, `POST /api/search_interactions`, `POST /api/search_user_playbooks`, `POST /api/search_agent_playbooks`, `GET /api/storage_stats`, `GET /api/get_profile_statistics`, `POST /api/get_dashboard_stats`, `POST /api/get_playbook_application_stats`
+- **Search/stats/provenance**: `POST /api/search`, `POST /api/search_profiles`, `POST /api/rerank_user_profiles`, `POST /api/search_interactions`, `POST /api/search_user_playbooks`, `POST /api/search_agent_playbooks`, `POST /api/get_learning_provenance`, `GET /api/storage_stats`, `GET /api/get_profile_statistics`, `POST /api/get_dashboard_stats`, `POST /api/get_playbook_application_stats`
 - **Profile lifecycle**: `POST /api/rerun_profile_generation`, `POST /api/manual_profile_generation`, `POST /api/upgrade_all_profiles`, `POST /api/downgrade_all_profiles`, `GET /api/profile_change_log`, `PUT /api/update_user_profile`, `DELETE /api/delete_profile`, `DELETE /api/delete_profiles_by_ids`, `DELETE /api/delete_all_profiles`
 - **Playbook lifecycle**: `POST /api/rerun_playbook_generation`, `POST /api/manual_playbook_generation`, `POST /api/run_playbook_aggregation`, `GET /api/playbook_aggregation_change_logs`, `POST /api/upgrade_all_user_playbooks`, `POST /api/downgrade_all_user_playbooks`, `PUT /api/update_agent_playbook_status`, `PUT /api/update_agent_playbook`, `PUT /api/update_user_playbook`, `DELETE /api/delete_agent_playbook`, `DELETE /api/delete_user_playbook`, `DELETE /api/delete_agent_playbooks_by_ids`, `DELETE /api/delete_user_playbooks_by_ids`, `DELETE /api/delete_all_playbooks`, `DELETE /api/delete_all_user_playbooks`, `DELETE /api/delete_all_agent_playbooks`
 - **Evaluation**: `POST /api/get_evaluation_overview`, `POST /api/evaluations/regenerate`, `GET /api/evaluations/regenerate/{job_id}`, `DELETE /api/evaluations/regenerate/{job_id}`, `POST /api/evaluations/grade_on_demand`, `GET /api/evaluations/shadow_comparisons/recent`
@@ -185,7 +185,8 @@ python -m reflexio.server.scripts.manage_invitation_codes list --show-used
 - **Playbook memory**: `playbook/` extracts user playbooks, consolidates them against existing rows, aggregates them into agent playbooks, and tracks aggregation change logs.
 - **Evaluation**: `agent_success_evaluation/service.py`, `agent_success_evaluation/runner.py`, `agent_success_evaluation/scheduler.py`, `agent_success_evaluation/components/evaluator.py`, `shadow_comparison/`, and `evaluation_overview/` handle session grading, per-turn shadow verdicts, regeneration jobs, and dashboard-facing rollups.
 - **Async clarification**: `extraction/` and `reflection/` manage resumable agent runs, pending tool calls, prior-answer search, and long-horizon reflection updates.
-- **Search preparation**: `pre_retrieval/` and `unified_search_service.py` handle query reformulation, document expansion, embeddings, and cross-entity search orchestration.
+- **Search preparation**: `pre_retrieval/`, `retrieval/`, and `unified_search_service.py` handle query reformulation, document expansion, relevance/recency ranking, embeddings, and cross-entity search orchestration.
+- **Governance**: `governance/` coordinates user export/erasure, audit events, privacy-safe subject references, and idempotent purge/rebuild workflows over storage governance primitives.
 - **Optimization/integrations**: `playbook_optimizer/` and `braintrust/` run candidate playbook optimization, rollout support, and Braintrust export/sync.
 - **Lineage**: `lineage/` resolves active records across superseded chains and schedules tombstone garbage collection for profile/playbook storage.
 - **Persistence/config**: `storage/`, `configurator/`, and `operation_state_utils.py` provide storage abstractions, config loading, locks, bookmarks, progress, and cancellation.
@@ -477,8 +478,8 @@ Pre-computed embeddings passed to storage methods via `query_embedding` paramete
 
 | File | Purpose |
 |------|---------|
-| `storage_base/` | BaseStorage interface split by domain (`_profiles.py`, `_playbook.py`, `_requests.py`, `_operations.py`, `_agent_run.py`, `_lineage.py`, `_shadow_verdicts.py`, `_stall_state.py`, `_share_links.py`) |
-| `sqlite_storage/` | SQLite-backed implementation split across the same domains, including lineage/tombstone support in `_lineage.py` |
+| `storage_base/` | BaseStorage interface split by domain (`_profiles.py`, `_playbook.py`, `_requests.py`, `_operations.py`, `_agent_run.py`, `_governance.py`, `_lineage.py`, `_shadow_verdicts.py`, `_stall_state.py`, `_share_links.py`) |
+| `sqlite_storage/` | SQLite-backed implementation split across the same domains, including governance/audit tables in `_governance.py` and lineage/tombstone support in `_lineage.py` |
 | `retention.py`, `retention_mixin.py` | Data retention and cleanup helpers |
 | `constants.py`, `error.py` | Storage constants and shared errors |
 
@@ -630,6 +631,7 @@ All services follow BaseGenerationService:
 - [Code Map (root README)](../README.md) -- high-level overview of all Reflexio components
 - [API Endpoints README](api_endpoints/README.md) -- RequestContext contract and handler/helper map
 - [Services README](services/README.md) -- per-directory index of the business-logic layer
+- [Governance Service README](services/governance/README.md) -- user export/erasure, audit, and purge workflow map
 - [Prompt Bank README](prompt/prompt_bank/README.md) -- versioned prompt template system
 - [Playbook Service README](services/playbook/README.md) -- playbook extraction, aggregation, and deduplication pipeline
 - [Site Variables README](site_var/README.md) -- global configuration and feature flags
diff --git a/reflexio/server/api_endpoints/README.md b/reflexio/server/api_endpoints/README.md
@@ -25,7 +25,8 @@ api.py (core_router + sub-routers)
 
 - **`RequestContext` is the context-passing contract** — handlers receive it via `Depends` and never reach for storage/config/prompts globally.
 - **Route handlers call `Reflexio` through `get_reflexio(org_id)`** — these helper files do **not** instantiate `Reflexio()` directly.
-- **Auth is injected, not implemented here** — the OS app uses `default_get_org_id` / `DEFAULT_ORG_ID`; the enterprise extension swaps in authenticated org resolution (see `reflexio_ext/server/api_endpoints/`).
+- **Learning provenance currently lives in `../api.py`** — `POST /api/get_learning_provenance` reconstructs provenance for profiles/user playbooks/agent playbooks from storage and generation metadata.
+- **Auth is injected, not implemented here** — the OSS app uses `default_get_org_id` / `DEFAULT_ORG_ID`; the enterprise extension swaps in authenticated org resolution (see `reflexio_ext/server/api_endpoints/`).
 
 ## Requirements / Problems to Avoid
 

diff --git a/reflexio/server/services/README.md b/reflexio/server/services/README.md
@@ -51,17 +51,18 @@ strings before deleting old import paths in the same PR.
 | `evaluation_overview/` | Dashboard/read-side rollups: `service.py` entry point, `components/` aggregation helpers, and root `eval_sampler.py` shared with regenerate jobs. See [README](evaluation_overview/README.md). |
 | `playbook_optimizer/` | Scenario-based playbook optimization: mature flat package with `optimizer.py`, `scheduler.py`, `rollout.py`, `judge.py`, `models.py`, `scenario_resolver.py`, `gepa_adapter.py`, and `assistant_webhook.py`. See [README](playbook_optimizer/README.md). |
 | `braintrust/` | Braintrust export/sync: `service.py`, `client.py`, `_cron.py`, `_encryption.py`. |
+| `governance/` | `GovernanceService` for user export/erasure, audit events, privacy-safe subject references, and idempotent purge workflows. See [README](governance/README.md). |
 | `lineage/` | Current-record resolution and tombstone GC: `resolve.py`, `gc_scheduler.py`. |
 | `pre_retrieval/` | `QueryReformulator` (`_query_reformulator.py`) + `DocumentExpander` (`_document_expander.py`) - query rewrite and doc expansion for recall. Compact by design; see [README](pre_retrieval/README.md). |
 | `tagging/` | `TaggingService` (`service.py`) + deferred `tagging_scheduler.py` - post-generation profile/playbook tagging. Compact by design; see [README](tagging/README.md). |
 | `unified_search_service.py` | `run_unified_search()` — two-phase parallel search across profiles / agent playbooks / user playbooks. |
-| `retrieval/` | `relevance_floor.py` — result relevance thresholding. |
+| `retrieval/` | `relevance_floor.py` and `recency.py` — result relevance thresholding plus recency-aware ranking helpers. |
 
 ## Persistence & Config
 
 | Path | Purpose |
 |------|---------|
-| `storage/` | `storage_base/` (`BaseStorage` split by domain, including `_lineage.py`) + `sqlite_storage/` (including lineage/tombstones) + `retention*.py`. Access via `request_context.storage` only. |
+| `storage/` | `storage_base/` (`BaseStorage` split by domain, including `_governance.py` and `_lineage.py`) + `sqlite_storage/` (including governance/audit tables and lineage/tombstones) + `retention*.py`. Access via `request_context.storage` only. |
 | `configurator/` | `DefaultConfigurator` — loads YAML config and creates the storage backend. |
 
 ## Key Rules

diff --git a/reflexio/server/services/governance/README.md b/reflexio/server/services/governance/README.md
@@ -0,0 +1,40 @@
+# server/services/governance
+Description: Governance service layer for user data export, erasure, audit events, and idempotent purge operation tracking.
+
+## Main Entry Points
+
+| File | Purpose |
+|------|---------|
+| `service.py` | `GovernanceService` orchestrates user export and user erasure workflows against `request_context.storage`. |
+| `subject_refs.py` | Stable privacy-safe reference helpers (`subject_ref`, `request_ref`, `stable_id`) used in audit and purge records. |
+
+## Purpose
+
+1. **User export** - Build a bundle of a user's profiles, interactions, requests, sessions, and user playbooks, then append an audit event.
+2. **User erasure** - Prepare idempotent purge targets, delete user-owned rows, hide/rebuild impacted agent playbooks, and complete or fail the purge with audit state.
+3. **Privacy-safe bookkeeping** - Store hashed subject/request references instead of raw user or request identifiers in governance audit/purge rows.
+
+## Architecture Pattern
+
+`GovernanceService` is a focused orchestration layer over storage primitives. It does not own persistence directly; backend-neutral contracts live in `storage/storage_base/_governance.py` and the SQLite implementation lives in `storage/sqlite_storage/_governance.py`.
+
+```
+GovernanceService
+  -> storage.begin_purge_operation / prepare_governance_erase_targets
+  -> storage.apply_governance_user_data_delete
+  -> storage.apply_governance_agent_playbook_rebuild
+  -> storage.complete_purge_operation_with_audit
+```
+
+## Key Contracts
+
+- `export_user(user_id, request_id) -> UserExportResult`
+- `erase_user(user_id, request_id) -> UserEraseResult`
+- Domain schemas: `models/api_schema/domain/governance.py`
+- Retention config: `Config.governance_retention` / `GovernanceRetentionConfig`
+
+## Requirements / Problems to Avoid
+
+- **Do not store raw identifiers in audit/purge references** — use `subject_refs.py` helpers.
+- **Do not bypass storage governance methods** — purge target preparation, deletion, rebuild, and audit completion are idempotency-sensitive.
+- **Keep governance storage backend-neutral** — add abstract methods to `storage_base/_governance.py` before implementation-specific code.