[M1.2.1] Design FlatBuffers schema for Mimi messages

Cargo.toml

@@ -26,6 +26,8 @@ log = "0.4"
 env_logger = "0.11"
 num_cpus = "1.16"
 tokio-test = "0.4"
+flatbuffers = "24.3"
+flatbuffers-build = "24.3"
 
 [profile.dev]
 opt-level = 0

build.rs

@@ -0,0 +1,15 @@
+use std::path::PathBuf;
+
+fn main() {
+    let out_dir = PathBuf::from(std::env::var("OUT_DIR").unwrap());
+
+    // Compile FlatBuffers schemas to Rust
+    flatbuffers_build::compile_flatbuffers_files(
+        &["proto/schema.fbs", "proto/metadata.fbs"],
+        &["proto/"],
+        &out_dir,
+    )
+    .expect("Failed to compile FlatBuffers schemas");
+
+    println!("cargo:rerun-if-changed=proto/");
+}

docs/M1.2.1-FLATBUFFERS-SCHEMA.md

@@ -0,0 +1,297 @@
+# M1.2.1: FlatBuffers Message Schema
+
+## Overview
+
+This document describes the comprehensive FlatBuffers schema (`proto/schema.fbs`) that defines all inter-module message types for the MiMi cognitive architecture. The schema is the **single source of truth** for message serialization/deserialization across Rust and C++ components communicating via the Zenoh Message Bus.
+
+## Design Principles
+
+### 1. Zero-Copy Deserialization
+- All messages use FlatBuffers for direct, zero-copy access to binary data
+- No intermediate allocation or unpacking needed
+- Enables sub-millisecond deserialization latency
+
+### 2. Type Safety & Versioning
+- Enum-based message types prevent runtime type confusion
+- Protocol version field enables backward compatibility
+- Union types ensure type-safe polymorphism
+
+### 3. Trace Context & Observability
+- Every message carries `TraceContext` for distributed tracing
+- Correlation IDs link requests across the entire processing pipeline
+- Latency tracking per message hop
+
+### 4. Budget-Aware Routing
+- Priority levels (CRITICAL, HIGH, MEDIUM, LOW) for gating system
+- Complexity estimates guide routing decisions (Tier 1/2/3)
+- Token cost tracking for budget management
+
+### 5. Semantic Integrity
+- Tone profiles capture emotional/voice metadata (Liliana)
+- Mood state embedded in responses
+- Security flags from Odlaguna gate included in messages
+
+## Schema Structure
+
+### Core Enumerations
+
+#### `IntentType` (Beatrice Classification)
+- QUERY: Information request
+- ACTION: Task execution request
+- SKILL_CREATION: System-generated skill
+- RESPONSE: Reply to prior message
+- ERROR: Error notification
+- EVENT: System event (state change)
+- CONTROL: Administrative control
+
+#### `ComplexityEstimate` (Gating System)
+- TRIVIAL: Cache hit, <50ms (Tier 1 Liliana)
+- SOCIAL: Greeting/small-talk (Tier 1 Liliana)
+- SIMPLE: Known skill, <200ms (Tier 2 Beatrice+Echidna)
+- MODERATE: Skill + light reasoning (Tier 2)
+- COMPLEX: Full cognitive pipeline (Tier 3)
+
+#### `GatingTier` (Routing Decision)
+- TIER_1_REFLEX: Liliana cache + template
+- TIER_2_AUTOMATED: Beatrice + Echidna skill
+- TIER_3_COGNITIVE: Full pipeline (Priscilla + Pandora + Echidna + LLM)
+
+#### `ResponseStatus` (Execution Outcome)
+- SUCCESS: Task completed successfully
+- PARTIAL: Partial completion (timeout, partial results)
+- FAILURE: Task failed
+- TIMEOUT: Execution timeout exceeded
+- REJECTED: Rejected by Odlaguna gate
+
+#### `CacheLayer` (Liliana Cache Classification)
+- NONE: Cache miss
+- EXACT_MATCH: Exact cache hit
+- SEMANTIC: Semantic similarity match
+- TEMPLATE: Template-based response
+
+### Message Types
+
+#### User Input (Beatrice → Mimi)
+```
+UserInput {
+  user_message: string,        // Raw user text
+  user_id: string,             // User identifier
+  session_id: string,          // Conversation session
+  channel: string              // Input channel (cli, http, etc.)
+}
+```
+
+#### Intent Classification (Beatrice → Mimi/Liliana)
+```
+IntentClassified {
+  intent_type: IntentType,
+  confidence: float,           // [0,1] classification confidence
+  entity_tags: [string],       // Extracted entities
+  is_social: bool,
+  estimated_complexity: ComplexityEstimate,
+  raw_intent_text: string
+}
+```
+
+#### Liliana Response (Liliana → Odlaguna)
+```
+LilianaResponse {
+  response: string,
+  mood: Mood,                  // Current mood state
+  confidence: float,           // Response appropriateness confidence
+  tone_profile: ToneProfile,   // Voice/personality metadata
+  cached_response: CachedResponse,
+  fallback_metadata: string    // JSON fallback hint
+}
+```
+
+#### Task Execution (Mimi → Ryzu)
+```
+TaskExecution {
+  task_id: string,
+  skill_id: string,
+  skill_name: string,
+  priority: Priority,
+  parameters: [byte],          // Serialized skill params
+  timeout_ms: uint32,
+  routing_decision: RoutingDecision
+}
+```
+
+#### Execution Result (Ryzu → Mimi)
+```
+ExecutionResult {
+  task_id: string,
+  success: bool,
+  status: ResponseStatus,
+  output: string,
+  error: string,
+  execution_time_ms: uint32,
+  tokens_consumed: uint32      // For budget tracking
+}
+```
+
+#### Memory Update (Mimi → Pandora)
+```
+MemoryUpdate {
+  update_id: string,
+  entity_id: string,           // Neo4j node ID
+  action: string,              // create, update, link, delete
+  entity_type: string,         // concept, skill, interaction
+  properties: string,          // JSON
+  heatmap_decay: float,        // Thermal decay [0,1]
+  retention_priority: Priority
+}
+```
+
+#### Odlaguna Gate (Odlaguna → Liliana)
+```
+OdalgunaGate {
+  message_id: string,
+  allowed: bool,
+  modifications: [Modification],  // Suggested changes
+  security_flags: SecurityFlags,
+  confidence: float,           // Gate confidence [0,1]
+  latency_ms: uint32
+}
+```
+
+#### Error Report (Any Module → Odlaguna)
+```
+ErrorReport {
+  error_id: string,
+  severity: Priority,
+  module: string,              // Where error occurred
+  error_message: string,
+  error_type: string,          // Classification
+  stack_trace: string,         // Debug-only
+  context_json: string         // Additional context
+}
+```
+
+### Unified Message Envelope
+
+All messages wrap their payload in a `Message` type:
+
+```
+Message {
+  version: uint32,             // Protocol version
+  trace: TraceContext,         // Distributed tracing
+  body_type: MessageBodyType,  // Discriminant
+  body: MessageBody,           // Union of all message types
+  routing_hint: string         // Target topic/route hint
+}
+```
+
+**Root type:** `Message`
+
+## Code Generation
+
+### Rust
+
+**Build Script** (`build.rs`):
+```rust
+flatbuffers_build::compile_flatbuffers_files(
+    &["proto/schema.fbs", "proto/metadata.fbs"],
+    &["proto/"],
+    &out_dir,
+).expect("Failed to compile FlatBuffers schemas");
+```
+
+**Output:** Rust bindings generated to `$OUT_DIR/mimi_protocol.rs`
+
+**Usage:**
+```rust
+use mimi::protocol::*;
+use flatbuffers::FlatBufferBuilder;
+
+let mut builder = FlatBufferBuilder::new(1024);
+let mut msg_builder = builders::MessageBuilder::new(&mut builder);
+let intent = msg_builder.create_intent_classified(
+    IntentType::QUERY,
+    0.85,
+    false,
+    ComplexityEstimate::SIMPLE
+);
+```
+
+**Validation:**
+```rust
+use mimi::protocol::validation::*;
+
+validate_intent_classified(&intent)?;
+validate_mood(&mood)?;
+```
+
+### C++
+
+**Generation Script** (`scripts/generate_cpp_bindings.py`):
+```bash
+python3 scripts/generate_cpp_bindings.py
+```
+
+**Output:** C++ headers generated to `proto/generated/cpp/`
+
+**Usage:**
+```cpp
+#include "proto/generated/cpp/schema_generated.h"
+using namespace MiMi::Protocol;
+
+// Access zero-copy data
+auto message = GetMessage(buffer);
+auto body = message->body_as_UserInput();
+```
+
+## Integration with Zenoh Topics
+
+Messages are published/subscribed on topic hierarchies mapped to message types:
+
+| Message Type | Zenoh Topic | Direction |
+|---|---|---|
+| UserInput | `beatrice/input` | Beatrice → Mimi |
+| IntentClassified | `beatrice/intent` | Beatrice → Liliana/Mimi |
+| LilianaResponse | `liliana/response` | Liliana → Odlaguna |
+| TaskExecution | `mimi/task/execute` | Mimi → Ryzu |
+| ExecutionResult | `mimi/task/result` | Ryzu → Mimi |
+| MemoryUpdate | `pandora/memory/update` | Mimi → Pandora |
+| OdalgunaGate | `odlaguna/gate` | Odlaguna → Liliana |
+| ErrorReport | `sys/error` | Any → Odlaguna |
+| SkillPublish | `echidna/skill/publish` | Echidna → Odlaguna |
+| EventNotification | `sys/events` | Any → Bus |
+
+## Latency & Performance Targets
+
+| Operation | Budget |
+|---|---|
+| Serialization (Rust) | < 100μs |
+| Deserialization (zero-copy) | < 50μs |
+| Zenoh broker latency | < 1ms |
+| End-to-end message hop | < 5ms |
+
+## Backward Compatibility
+
+Protocol version field enables future evolution:
+- Version mismatch detection
+- Graceful downgrade/upgrade paths
+- Schema evolution without breaking existing deployments
+
+## Testing
+
+Unit tests included in `src/protocol.rs`:
+
+```bash
+cargo test --lib protocol::tests
+```
+
+Tests validate:
+- Message builder correctness
+- Mood/intent validation
+- Round-trip serialization
+- Boundary conditions
+
+## Future Extensions
+
+Reserved for M1.3+:
+- Compression options (for large skill binaries)
+- Streaming message batching
+- End-to-end encryption metadata

proto/metadata.fbs

@@ -0,0 +1,21 @@
+namespace MiMi.Protocol;
+
+table RuntimeMetrics {
+  start_time_ms: uint64;
+  end_time_ms: uint64;
+  duration_ms: uint32;
+  cpu_percent: float;
+  memory_mb: uint32;
+  tokens_input: uint32;
+  tokens_output: uint32;
+}
+
+table BudgetSnapshot {
+  tier1_used: uint32;
+  tier1_limit: uint32;
+  tier2_used: uint32;
+  tier2_limit: uint32;
+  tier3_used: uint32;
+  tier3_limit: uint32;
+  timestamp_ms: uint64;
+}