docs: Cookbook and Learning Path Improvements

docs/.agent/docs_coverage.md

@@ -1,32 +1,30 @@
 # Docs Coverage Map
 
-| Feature | Documentation | Code Location | Status |
-|---------|---------------|---------------|--------|
+| Feature Area | Documentation Page | Source Code (Key Symbols) | Status |
+|--------------|-------------------|--------------------------|--------|
 | **Core** | | | |
-| Routing | `concepts/handlers.md` | `rustapi-macros` | OK |
-| Extractors | `concepts/handlers.md` | `rustapi-core/src/extract.rs` | OK |
-| State | `concepts/handlers.md` | `rustapi-core/src/extract.rs` | OK |
-| Validation | `crates/rustapi_validate.md` | `rustapi-validate` | OK |
-| **HATEOAS** | | | |
-| Pagination | `recipes/pagination.md` | `rustapi-core/src/hateoas.rs` | OK |
-| Links | `recipes/pagination.md` | `rustapi-core/src/hateoas.rs` | OK |
+| Routing | `docs/cookbook/src/concepts/routing.md` | `rustapi-core/src/router.rs` (`Router`) | OK |
+| Handlers | `docs/cookbook/src/concepts/handlers.md` | `rustapi-core/src/handler.rs` (`Handler`) | OK |
+| Extractors | `docs/cookbook/src/concepts/extractors.md` | `rustapi-core/src/extract.rs` (`FromRequest`) | OK |
+| Middleware | `docs/cookbook/src/recipes/custom_middleware.md` | `rustapi-core/src/middleware/mod.rs` (`MiddlewareLayer`) | OK |
+| State | `docs/cookbook/src/concepts/state.md` | `rustapi-core/src/extract.rs` (`State`) | OK |
+| Error Handling | `docs/cookbook/src/concepts/errors.md` | `rustapi-core/src/error.rs` (`ApiError`) | OK |
+| HTTP/3 (QUIC) | `docs/cookbook/src/recipes/http3_quic.md` | `rustapi-core/src/http3.rs` (`Http3Server`) | OK |
+| File Uploads | `docs/cookbook/src/recipes/file_uploads.md` | `rustapi-core/src/multipart.rs` (`Multipart`) | OK |
+| Compression | `docs/cookbook/src/recipes/compression.md` | `rustapi-core/src/middleware/compression.rs` (`CompressionLayer`) | OK |
+| **OpenAPI** | | | |
+| Schema Derivation | `docs/cookbook/src/crates/rustapi_openapi.md` | `rustapi-macros/src/derive_schema.rs` (`#[derive(Schema)]`) | OK |
+| References ($ref) | `docs/cookbook/src/recipes/openapi_refs.md` | `rustapi-openapi/src/schema.rs` (`SchemaRef`) | OK |
+| **Validation** | | | |
+| Sync Validation | `docs/cookbook/src/crates/rustapi_validate.md` | `rustapi-validate/src/lib.rs` (`Validate`) | OK |
+| Async Validation | `docs/cookbook/src/crates/rustapi_validate.md` | `rustapi-validate/src/v2/mod.rs` (`AsyncValidate`) | OK |
 | **Extras** | | | |
-| Auth (JWT) | `recipes/jwt_auth.md` | `rustapi-extras/src/jwt` | OK |
-| Auth (OAuth2) | `recipes/oauth2_client.md` | `rustapi-extras/src/oauth2` | OK |
-| Security | `recipes/csrf_protection.md` | `rustapi-extras/src/security` | OK |
-| Observability | `crates/rustapi_extras.md` | `rustapi-extras/src/telemetry` | OK |
-| Audit Logging | `recipes/audit_logging.md` | `rustapi-extras/src/audit` | OK |
-| Middleware (Advanced) | `recipes/advanced_middleware.md` | `rustapi-extras/src/{rate_limit, dedup, cache}` | OK |
-| **Jobs** | | | |
-| Job Queue (Crate) | `crates/rustapi_jobs.md` | `rustapi-jobs` | OK |
-| Background Jobs (Recipe) | `recipes/background_jobs.md` | `rustapi-jobs` | OK |
-| **Integrations** | | | |
-| gRPC | `recipes/grpc_integration.md` | `rustapi-grpc` | OK |
-| SSR | `recipes/server_side_rendering.md` | `rustapi-view` | OK |
-| AI / TOON | `recipes/ai_integration.md` | `rustapi-toon` | OK |
-| **Learning** | | | |
-| Structured Path | `learning/curriculum.md` | N/A | OK |
-| **Recipes** | | | |
-| File Uploads | `recipes/file_uploads.md` | `rustapi-core` | OK |
-| Deployment | `recipes/deployment.md` | `cargo-rustapi` | OK |
-| Testing | `recipes/testing.md` | `rustapi-testing` | OK |
+| JWT Auth | `docs/cookbook/src/recipes/jwt_auth.md` | `rustapi-extras/src/jwt.rs` (`JwtLayer`) | OK |
+| OAuth2 | `docs/cookbook/src/recipes/oauth2_client.md` | `rustapi-extras/src/oauth2.rs` (`OAuth2Client`) | OK |
+| Database | `docs/cookbook/src/recipes/db_integration.md` | N/A (Integration pattern) | Needs Update |
+| **Ecosystem** | | | |
+| WebSockets | `docs/cookbook/src/recipes/websockets.md` | `rustapi-ws/src/lib.rs` (`WebSocketUpgrade`) | OK |
+| SSR (View) | `docs/cookbook/src/recipes/server_side_rendering.md` | `rustapi-view/src/lib.rs` (`View`) | OK |
+| gRPC | `docs/cookbook/src/recipes/grpc_integration.md` | `rustapi-grpc/src/lib.rs` (`TonicServer`) | OK |
+| Jobs | `docs/cookbook/src/recipes/background_jobs.md` | `rustapi-jobs/src/lib.rs` (`Job`) | OK |
+| TOON (AI) | `docs/cookbook/src/recipes/ai_integration.md` | `rustapi-toon/src/lib.rs` (`LlmResponse`) | OK |

docs/.agent/docs_inventory.md

@@ -1,15 +1,25 @@
 # Documentation Inventory
 
-| File | Purpose | Owner Crate | Status |
-|------|---------|-------------|--------|
-| `README.md` | Project overview, key features, quick start | Root | OK |
-| `docs/README.md` | Documentation landing page | Docs | OK |
-| `docs/cookbook/src/SUMMARY.md` | Cookbook navigation structure | Docs | OK |
-| `docs/cookbook/src/getting_started/installation.md` | Installation instructions | Docs | OK |
-| `docs/cookbook/src/learning/README.md` | Learning path entry point | Docs | OK |
-| `docs/cookbook/src/recipes/*.md` | Specific implementation guides | Docs | OK |
-| `docs/cookbook/src/recipes/advanced_middleware.md` | Recipe for Rate Limit, Dedup, Cache | Docs | OK |
-| `docs/cookbook/src/recipes/audit_logging.md` | Recipe for Audit Logging | Docs | OK |
-| `docs/cookbook/src/recipes/oauth2_client.md` | Recipe for OAuth2 Client | Docs | OK |
-| `crates/rustapi-core/src/hateoas.rs` | API Reference for HATEOAS | rustapi-core | OK |
-| `crates/rustapi-core/src/extract.rs` | API Reference for Extractors | rustapi-core | OK |
+| File Path | Purpose | Last Updated Version | Owner Crate | Status |
+|-----------|---------|----------------------|-------------|--------|
+| `docs/README.md` | Main entry point | 0.1.335 | rustapi-rs | OK |
+| `docs/GETTING_STARTED.md` | Quick start guide | 0.1.335 | rustapi-rs | OK |
+| `docs/ARCHITECTURE.md` | High-level architecture | 0.1.335 | rustapi-core | OK |
+| `docs/FEATURES.md` | Feature list | 0.1.335 | rustapi-rs | OK |
+| `docs/PHILOSOPHY.md` | Design philosophy | 0.1.335 | rustapi-rs | OK |
+| `docs/native_openapi.md` | Native OpenAPI details | 0.1.335 | rustapi-openapi | OK |
+| `docs/cookbook/src/SUMMARY.md` | Cookbook ToC | 0.1.335 | rustapi-rs | Needs Update |
+| `docs/cookbook/src/introduction.md` | Cookbook Intro | 0.1.335 | rustapi-rs | OK |
+| `docs/cookbook/src/troubleshooting.md` | Common issues | 0.1.335 | rustapi-rs | OK |
+| `docs/cookbook/src/learning/curriculum.md` | Learning Path | 0.1.335 | rustapi-rs | Needs Update |
+| `docs/cookbook/src/recipes/db_integration.md` | Database recipe | 0.1.335 | rustapi-rs | Needs Update |
+| `docs/cookbook/src/recipes/file_uploads.md` | File upload recipe | 0.1.335 | rustapi-core | OK |
+| `docs/cookbook/src/recipes/compression.md` | Compression recipe | 0.1.335 | rustapi-core | OK |
+| `docs/cookbook/src/recipes/openapi_refs.md` | OpenAPI Refs recipe | 0.1.335 | rustapi-openapi | OK |
+| `docs/cookbook/src/recipes/http3_quic.md` | HTTP/3 recipe | 0.1.335 | rustapi-core | OK |
+| `docs/cookbook/src/recipes/jwt_auth.md` | JWT Auth recipe | 0.1.335 | rustapi-extras | OK |
+| `docs/cookbook/src/recipes/websockets.md` | WebSocket recipe | 0.1.335 | rustapi-ws | OK |
+| `docs/cookbook/src/recipes/server_side_rendering.md` | SSR recipe | 0.1.335 | rustapi-view | OK |
+| `docs/cookbook/src/recipes/grpc_integration.md` | gRPC recipe | 0.1.335 | rustapi-grpc | OK |
+| `docs/cookbook/src/recipes/background_jobs.md` | Jobs recipe | 0.1.335 | rustapi-jobs | OK |
+| `docs/cookbook/src/recipes/ai_integration.md` | AI/TOON recipe | 0.1.335 | rustapi-toon | OK |

docs/.agent/last_run.json

@@ -1,5 +1,5 @@
 {
   "last_processed_ref": "v0.1.335",
-  "date": "2026-02-16",
-  "notes": "Added recipes for Advanced Middleware, Audit Logging, and OAuth2 Client. Updated Learning Path to include these features."
+  "date": "2026-02-17",
+  "notes": "Added recipes for Compression, OpenAPI Refs, File Uploads, and Database Integration. Updated Learning Path with new modules."
 }

docs/.agent/run_report_2026-02-17.md

@@ -0,0 +1,35 @@
+# Run Report: 2026-02-17
+
+## Detected Version
+- **Target Version:** 0.1.335
+- **Commit:** (No new commits since last run)
+
+## Changes
+No code changes detected. This run focused on documentation improvements and cookbook expansion.
+
+## Documentation Updates
+
+### Learning Path (`docs/cookbook/src/learning/curriculum.md`)
+- Added **Module 4.5: Database Integration** covering connection pooling with `sqlx`.
+- Added **Module 5.5: Error Handling** covering `ApiError` and production masking.
+- Added **Module 6.5: File Uploads & Multipart** covering streaming uploads.
+- Updated **Module 6: OpenAPI & HATEOAS** to include OpenAPI References.
+- Updated **Module 14: High Performance** to include Response Compression.
+
+### Cookbook Recipes
+- **Created `recipes/compression.md`:** Detailed guide on using `CompressionLayer` with Gzip/Brotli/Deflate.
+- **Created `recipes/openapi_refs.md`:** Explanation of automatic `$ref` generation with `#[derive(Schema)]` and handling recursive types.
+- **Updated `recipes/file_uploads.md`:** Fixed body limit configuration (using `RustApi::new().body_limit(...)`), improved security notes, and added a complete example.
+- **Updated `recipes/db_integration.md`:** Expanded with production connection pool settings, transactions, and integration testing with `testcontainers`.
+
+### Documentation Management
+- **Created `docs/.agent/docs_inventory.md`:** Full inventory of documentation files and their status.
+- **Created `docs/.agent/docs_coverage.md`:** Mapping of features to documentation pages.
+
+## Improvements
+- Addressed user feedback regarding missing recipes for DB integration patterns, file uploads, error types, OpenAPI refs, and compression.
+- Standardized the Learning Path structure.
+
+## TODOs
+- Verify `rustapi-grpc` examples with the latest `tonic` version.
+- Add a specific recipe for `rustapi-view` with HTMX.

docs/cookbook/src/SUMMARY.md

@@ -30,6 +30,7 @@
 - [Part IV: Recipes](recipes/README.md)
     - [Creating Resources](recipes/crud_resource.md)
     - [Pagination & HATEOAS](recipes/pagination.md)
+    - [OpenAPI & Schemas](recipes/openapi_refs.md)
     - [JWT Authentication](recipes/jwt_auth.md)
     - [OAuth2 Client](recipes/oauth2_client.md)
     - [CSRF Protection](recipes/csrf_protection.md)
@@ -43,6 +44,7 @@
     - [Server-Side Rendering (SSR)](recipes/server_side_rendering.md)
     - [AI Integration (TOON)](recipes/ai_integration.md)
     - [Production Tuning](recipes/high_performance.md)
+    - [Response Compression](recipes/compression.md)
     - [Resilience Patterns](recipes/resilience.md)
     - [Audit Logging](recipes/audit_logging.md)
     - [Time-Travel Debugging (Replay)](recipes/replay.md)

docs/cookbook/src/learning/curriculum.md

@@ -69,6 +69,18 @@ This curriculum is designed to take you from a RustAPI beginner to an advanced u
 2. Which extractor retrieves the application state?
 3. Why should you use `Arc` for shared state?
 
+### Module 4.5: Database Integration
+- **Prerequisites:** Module 4.
+- **Reading:** [Database Integration](../recipes/db_integration.md).
+- **Task:** Replace the in-memory `Mutex<Vec<User>>` with a PostgreSQL connection pool (`sqlx::PgPool`).
+- **Expected Output:** Data persists across server restarts.
+- **Pitfalls:** Blocking the async runtime with synchronous DB drivers (use `sqlx` or `tokio-postgres`).
+
+#### 🧠 Knowledge Check
+1. Why is connection pooling important?
+2. How do you share a DB pool across handlers?
+3. What is the benefit of compile-time query checking in SQLx?
+
 ### Module 5: Validation
 - **Prerequisites:** Module 4.
 - **Reading:** [Validation](../crates/rustapi_validation.md).
@@ -81,18 +93,42 @@ This curriculum is designed to take you from a RustAPI beginner to an advanced u
 2. What HTTP status code is returned on validation failure?
 3. How do you combine JSON extraction and validation?
 
+### Module 5.5: Error Handling
+- **Prerequisites:** Module 5.
+- **Reading:** [Error Handling](../concepts/errors.md).
+- **Task:** Create a custom `ApiError` enum and implement `IntoResponse`. Return robust error messages.
+- **Expected Output:** `GET /users/999` returns `404 Not Found` with a structured JSON error body.
+- **Pitfalls:** Exposing internal database errors (like SQL strings) to the client.
+
+#### 🧠 Knowledge Check
+1. What is the standard error type in RustAPI?
+2. How do you mask internal errors in production?
+3. What is the purpose of the `error_id` field?
+
 ### Module 6: OpenAPI & HATEOAS
 - **Prerequisites:** Module 5.
-- **Reading:** [OpenAPI](../crates/rustapi_openapi.md), [Pagination Recipe](../recipes/pagination.md).
-- **Task:** Add `#[derive(Schema)]` to all DTOs. Implement pagination for `GET /users`.
-- **Expected Output:** Swagger UI at `/docs` showing full schema. Paginated responses with `_links`.
-- **Pitfalls:** Using types that don't implement `Schema` (like raw `serde_json::Value`) inside response structs.
+- **Reading:** [OpenAPI](../crates/rustapi_openapi.md), [OpenAPI Refs](../recipes/openapi_refs.md), [Pagination Recipe](../recipes/pagination.md).
+- **Task:** Add `#[derive(Schema)]` to all DTOs. Use `#[derive(Schema)]` on a shared struct and reference it in multiple places.
+- **Expected Output:** Swagger UI at `/docs` showing full schema with shared components.
+- **Pitfalls:** Recursive schemas without `Box` or `Option`.
 
 #### 🧠 Knowledge Check
 1. What does `#[derive(Schema)]` do?
-2. Where is the Swagger UI served by default?
+2. How does RustAPI handle shared schema components?
 3. What is HATEOAS and why is it useful?
 
+### Module 6.5: File Uploads & Multipart
+- **Prerequisites:** Module 6.
+- **Reading:** [File Uploads](../recipes/file_uploads.md).
+- **Task:** Create an endpoint `POST /upload` that accepts a file and saves it to disk.
+- **Expected Output:** `curl -F file=@image.png` uploads the file.
+- **Pitfalls:** Loading large files entirely into memory (use streaming).
+
+#### 🧠 Knowledge Check
+1. Which extractor is used for file uploads?
+2. Why should you use `field.chunk()` instead of `field.bytes()`?
+3. How do you increase the request body size limit?
+
 ### 🏆 Phase 2 Capstone: "The Secure Blog Engine"
 **Objective:** Enhance the Todo API into a Blog Engine.
 **Requirements:**
@@ -224,18 +260,18 @@ This curriculum is designed to take you from a RustAPI beginner to an advanced u
 
 ### Module 14: High Performance
 - **Prerequisites:** Phase 3.
-- **Reading:** [HTTP/3 (QUIC)](../recipes/http3_quic.md), [Performance Tuning](../recipes/high_performance.md).
+- **Reading:** [HTTP/3 (QUIC)](../recipes/http3_quic.md), [Performance Tuning](../recipes/high_performance.md), [Compression](../recipes/compression.md).
 - **Task:**
     1. Enable `http3` feature and generate self-signed certs.
     2. Serve traffic over QUIC.
-    3. Implement response caching for a heavy computation endpoint.
-- **Expected Output:** Browser/Client connects via HTTP/3. Repeated requests are served instantly from cache.
-- **Pitfalls:** Caching private user data without proper keys.
+    3. Add `CompressionLayer` to compress large responses.
+- **Expected Output:** Browser/Client connects via HTTP/3. Responses have `content-encoding: gzip`.
+- **Pitfalls:** Compressing small responses (waste of CPU) or already compressed data (images).
 
 #### 🧠 Knowledge Check
 1. What transport protocol does HTTP/3 use?
 2. How does `simd-json` improve performance?
-3. When should you *not* use caching?
+3. Why shouldn't you compress JPEG images?
 
 ### 🏆 Phase 4 Capstone: "The High-Scale Event Platform"
 **Objective:** Architect a system capable of handling thousands of events per second.

docs/cookbook/src/recipes/README.md

@@ -13,6 +13,7 @@ Each recipe follows a simple structure:
 
 - [Creating Resources](crud_resource.md)
 - [Pagination & HATEOAS](pagination.md)
+- [OpenAPI & Schemas](openapi_refs.md)
 - [JWT Authentication](jwt_auth.md)
 - [CSRF Protection](csrf_protection.md)
 - [Database Integration](db_integration.md)
@@ -24,6 +25,7 @@ Each recipe follows a simple structure:
 - [Server-Side Rendering (SSR)](server_side_rendering.md)
 - [AI Integration (TOON)](ai_integration.md)
 - [Production Tuning](high_performance.md)
+- [Response Compression](compression.md)
 - [Resilience Patterns](resilience.md)
 - [Time-Travel Debugging (Replay)](replay.md)
 - [Deployment](deployment.md)

docs/cookbook/src/recipes/compression.md

@@ -0,0 +1,87 @@
+# Response Compression
+
+RustAPI supports automatic response compression (Gzip, Deflate, Brotli) via the `CompressionLayer`. This middleware negotiates the best compression algorithm based on the client's `Accept-Encoding` header.
+
+## Dependencies
+
+To use compression, you must enable the `compression` feature in `rustapi-core` (or `rustapi-rs`). For Brotli support, enable `compression-brotli`.
+
+```toml
+[dependencies]
+rustapi-rs = { version = "0.1.335", features = ["compression", "compression-brotli"] }
+```
+
+## Basic Usage
+
+The simplest way to enable compression is to add the layer to your application:
+
+```rust
+use rustapi_rs::prelude::*;
+use rustapi_core::middleware::CompressionLayer;
+
+#[tokio::main]
+async fn main() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
+    RustApi::new()
+        .layer(CompressionLayer::new())
+        .route("/", get(hello))
+        .run("127.0.0.1:8080")
+        .await
+}
+
+async fn hello() -> &'static str {
+    "Hello, World! This response will be compressed if the client supports it."
+}
+```
+
+## Configuration
+
+You can customize the compression behavior using `CompressionConfig`:
+
+```rust
+use rustapi_rs::prelude::*;
+use rustapi_core::middleware::{CompressionLayer, CompressionConfig};
+
+#[tokio::main]
+async fn main() -> Result<()> {
+    let config = CompressionConfig::new()
+        .min_size(1024)       // Only compress responses larger than 1KB
+        .level(6)             // Compression level (0-9)
+        .gzip(true)           // Enable Gzip
+        .deflate(false)       // Disable Deflate
+        .brotli(true)         // Enable Brotli (if feature enabled)
+        .add_content_type("application/custom-json"); // Add custom type
+
+    RustApi::new()
+        .layer(CompressionLayer::with_config(config))
+        .route("/data", get(get_large_data))
+        .run("127.0.0.1:8080")
+        .await
+}
+```
+
+## Default Configuration
+
+By default, `CompressionLayer` is configured with:
+- `min_size`: 1024 bytes (1KB)
+- `level`: 6
+- `gzip`: enabled
+- `deflate`: enabled
+- `brotli`: enabled (if feature is present)
+- `content_types`: `text/*`, `application/json`, `application/javascript`, `application/xml`, `image/svg+xml`
+
+## Best Practices
+
+### 1. Don't Compress Already Compressed Data
+Images (JPEG, PNG), Videos, and Archives (ZIP) are already compressed. Compressing them again wastes CPU cycles and might even increase the file size. The default configuration excludes most binary formats, but be careful with custom types.
+
+### 2. Set Minimum Size
+Compressing very small responses (e.g., "OK") can actually make them larger due to framing overhead. The default 1KB threshold is a good starting point.
+
+### 3. Order of Middleware
+Compression should usually be one of the *last* layers added (outermost), so it compresses the final response after other middleware (like logging or headers) have run.
+
+```rust
+RustApi::new()
+    .layer(CompressionLayer::new()) // Runs last on response (first on request)
+    .layer(LoggingLayer::new())     // Runs before compression on response
+```