[trainer,data,model] feat: add SFT and offline DPO

.agents/knowledge/README.md

@@ -7,13 +7,18 @@
 | Touching dtype/precision, mixed precision config, debugging NaN/overflow | `topics/dtype_precision.md` |
 | Editing a trainer `optimize()` loop / autocast scope, ref/EMA/named param swaps | `topics/autocast_param_swap.md` |
 | Adding or modifying a model adapter | `topics/adapter_conventions.md` |
+| Changing offline `PipelineIOContract`, condition-state preparation, output codecs, or output geometry | `topics/adapter_conventions.md`, `topics/component_runtime.md` |
 | Changing component discovery, loading, lifecycle, or distributed preparation | `topics/component_runtime.md` |
+| Changing FSDP loading or activation-checkpoint ownership | `topics/component_runtime.md` |
 | Touching rollout collection, replay bridges, index maps, or multi-component order | `topics/structured_trajectory.md` |
 | Adding an algorithm that trains several model copies at once, or changing per-variant LoRA/full storage or variant checkpointing | `topics/component_variants.md` |
+| Changing optimizer roles, Muon, `CompositeOptimizer`, or optimizer/backend compatibility | `topics/component_variants.md`, `dependencies.md` |
 | Adding adapter, upgrading diffusers, debugging output quality | `topics/parity_testing.md` |
+| Touching MiniMax H3 workflows, dependency pins, ordered references, or H3 memory policies | `topics/minimax_h3.md` |
 | Touching `TimeSampler`, `adapter.forward(t=...)`, `timestep_range`, `flow_match_sigma` | `topics/timestep_sigma.md` |
 | Editing `data_utils/sampler*`, hparams sampler/batch fields | `topics/samplers.md` |
 | Touching `sample()`/`optimize()` data flow, debugging `sample()`/`optimize()` OOM, adding high-resolution / video example configs | `topics/sample_lifecycle.md` |
+| Changing `BaseSample`, partial sample gathering, or concrete-sample reconstruction | `topics/sample_lifecycle.md` |
 | After completing a bug fix | `topics/fix_patterns.md` |
 | Changing `pyproject.toml`, deps, install commands | `dependencies.md` |
 | Adding or editing `.agents/` documentation | `docs_maintenance.md` |

.agents/knowledge/architecture.md

@@ -21,7 +21,7 @@
    └──┬───┬───┬───┘  └──┬───┬───┬──┘  └──┬───┬───┬───┘
       │   │   │         │   │   │         │   │   │
       ▼   ▼   ▼         ▼   ▼   ▼         ▼   ▼   ▼
-    GRPO NFT AWM     Flux SD3 Wan     PickScore CLIP OCR
+    GRPO SFT DPO     Flux SD3 Wan     PickScore CLIP OCR
 ```
 
 ### Key Dependency Rules
@@ -42,51 +42,30 @@
 
 ---
 
-## Six-Stage Training Pipeline
+## Execution Pipelines
 
 > Authoritative reference: `guidance/workflow.md`
 
-```
-Stage 1: Data Preprocessing (offline, cached)
-  │  GeneralDataset + adapter.preprocess_func()
-  │  Text/image/video/audio → encoded tensors (prompt_embeds, image_latents, audio_features, ...)
-  │  Result cached with hash fingerprint
-  ▼
-Stage 2: K-Repeat Sampling
-  │  Three sampler strategies (see `topics/samplers.md`):
-  │  - GroupContiguousSampler (preferred, auto-selected): keeps K copies on same rank
-  │  - DistributedKRepeatSampler (fallback): shuffles K copies across ranks
-  │  - GroupDistributedSampler (DGPO): rank-identical prompt sequence, K/W copies per rank
-  │  K = training_args.group_size
-  ▼
-Stage 3: Trajectory Generation
-  │  adapter.inference() — full multi-step SDE/ODE denoising
-  │  Produces: generated images/videos + trajectory data (noises, log-probs)
-  ▼
-Stage 4: Reward Computation
-  │  RewardProcessor dispatches to Pointwise or Groupwise models
-  │  Multi-reward aggregation with configurable weights
-  ▼
-Stage 5: Advantage Computation
-  │  AdvantageProcessor (advantage/advantage_processor.py)
-  │  Communication-aware: auto-selects gather vs local path
-  │  Strategies: "sum" (weighted-sum, GRPO) or "gdpo"
-  ▼
-Stage 6: Policy Optimization
-  │  adapter.forward() — single-step denoising for loss computation
-  │  Policy gradient (GRPO) or weighted matching (NFT/AWM) or DPO preference loss
-  │  Gradient update via accelerator
-  ▼
-  (Repeat Stages 2–6 for next epoch)
-```
+`ExecutionContract` separates acquisition (`generation` or `dataset`) from feedback
+(`runtime_reward` or `none`). `PipelineIOContract` independently owns model input/output media,
+rates, geometry, and batching.
+
+| Composition | Driver | Optimization entry | Cycle counter |
+|---|---|---|---|
+| Generation + runtime reward | K-repeat → inference → reward → advantage | `optimize(samples)` | `rollout_iteration` |
+| Generation + no feedback | Generation/distillation path | `optimize(samples)` | `rollout_iteration` |
+| Dataset + no feedback | Finite official `DistributedSampler` traversal | `optimize_batch(batch)` | `data_epoch` |
 
-**Trainer methods vs stages** (each epoch, after Stage 1):
+`optimizer_step` advances independently. A dataset epoch advances only after clean loader
+exhaustion; offline output media is decoded and encoded on the fly, while only prompt/input
+conditions enter the preprocessing cache.
 
-| Method | Stages |
-|--------|--------|
-| `sample()` | 2–3 (K-repeat batches + `adapter.inference` trajectories) |
-| `prepare_feedback()` | 4–5: reward buffer finalize, `AdvantageProcessor` |
-| `optimize()` | 6: `adapter.forward` and optimizer step (DPO: form chosen/rejected pairs at entry, then loss) |
+Exact runtime identity is built from realized prepared state. It locks optimizer/model/backend
+semantics, the checkpoint-realized pipeline I/O contract, ordered training data, and the complete
+replayed evaluation path (cadence, arguments,
+per-dataset overrides, rewards, and ordered prepared loaders). Logging, checkpoint cadence, run
+budget, and resume location remain operational. Exact-state save fails before mutation on MPS
+because Accelerate does not persist the device RNG needed for exact continuation.
 
 ---
 
@@ -100,6 +79,8 @@ All four registries map string keys → lazy import paths. Resolution: registry
 
 | Key | Class | Paradigm | Base Class |
 |-----|-------|----------|------------|
+| `sft` | `SFTTrainer` | Decoupled, dataset | `BaseTrainer` |
+| `offline-dpo` | `OfflineDPOTrainer` | Decoupled, dataset | `BaseTrainer` |
 | `grpo` | `GRPOTrainer` | Coupled | `BaseTrainer` |
 | `grpo-guard` | `GRPOGuardTrainer` | Coupled | `GRPOTrainer` |
 | `dppo` | `DPPOTrainer` | Coupled | `GRPOTrainer` |
@@ -108,9 +89,14 @@ All four registries map string keys → lazy import paths. Resolution: registry
 | `nft` | `DiffusionNFTTrainer` | Decoupled | `BaseTrainer` |
 | `awm` | `AWMTrainer` | Decoupled | `BaseTrainer` |
 | `crd` | `CRDTrainer` | Decoupled | `BaseTrainer` |
-| `diffusion-opd` | `DiffusionOPDTrainer` | Distillation (on-policy) | `BaseTrainer` |
+| `diffusion-opd` | `DiffusionOPDTrainer` | Distillation, generation + no feedback | `BaseTrainer` |
+| `dmd2` | `DMD2Trainer` | Distillation, generation + no feedback, ODE | `BaseTrainer` |
+| `tdm` | `TDMTrainer` | Distillation, generation + no feedback, ODE | `BaseTrainer` |
+| `tdm-r1` | `TDMR1Trainer` | Decoupled, generation + runtime reward, ODE | `TDMTrainer` |
 
-**Flat hierarchy**: New trainers inherit from `BaseTrainer` directly. The sanctioned exceptions are `GRPOGuardTrainer → GRPOTrainer` and `DPPOTrainer → GRPOTrainer` (strict GRPO loss variants; see constraint #11).
+**Flat hierarchy**: New trainers inherit from `BaseTrainer` directly. The sanctioned existing
+extensions are `GRPOGuardTrainer → GRPOTrainer`, `DPPOTrainer → GRPOTrainer`, and
+`TDMR1Trainer → TDMTrainer`; see constraint #11.
 
 **Model Adapters** (`models/registry.py`):
 | Key | Class | Task |
@@ -125,11 +111,13 @@ All four registries map string keys → lazy import paths. Resolution: registry
 | `z-image` | `ZImageAdapter` | Text-to-Image |
 | `wan2_t2v` | `Wan2_T2V_Adapter` | Text-to-Video |
 | `wan2_i2v` | `Wan2_I2V_Adapter` | Image-to-Video |
-| `wan2_v2v` | `Wan2_V2V_Adapter` | Video-to-Video |
 | `ltx2_t2av` | `LTX2_T2AV_Adapter` | Text-to-Audio-Video |
 | `ltx2_i2av` | `LTX2_I2AV_Adapter` | Image-to-Audio-Video |
+| `minimax-h3-t2va` | `MiniMaxH3T2VAAdapter` | Text-to-Video-Audio |
+| `minimax-h3-fl2va` | `MiniMaxH3FL2VAAdapter` | Sparse First/Last-Frame-to-Video-Audio |
+| `minimax-h3-ref2va` | `MiniMaxH3Ref2VAAdapter` | Ordered-Reference-to-Video-Audio |
 | `bagel` | `BagelAdapter` | Text-to-Image & Image(s)-to-Image (T2I & I2I both batched via NaViT packing; subset-round packing handles variable I2I reference-image count, no per-sample fallback — see `topics/adapter_conventions.md`) |
-| `sensenova` | `SenseNovaAdapter` | Text-to-Image & Image(s)-to-Image (SenseNova-U1 1.0/1.5; ordered variable-count references; independent samples use B=1 prefixes rather than Bagel-style NaViT packing) |
+| `sensenova` | `SenseNovaAdapter` | Text-to-Image & Image(s)-to-Image (SenseNova-U1 1.0/1.5; ordered variable-count references remain grouped in `images` and preserve within-type order; independent samples use B=1 prefixes rather than Bagel-style NaViT packing) |
 
 **Reward Models** (`rewards/registry.py`):
 | Key | Class | Type |
@@ -163,7 +151,7 @@ Configured via the `acceleration:` block (`hparams/acceleration_args.py`): two o
 
 - **New model adapter**: `guidance/new_model.md`, skill `/ff-new-model`, conventions `topics/adapter_conventions.md`
 - **New reward model**: `guidance/rewards.md`, skill `/ff-new-reward`
-- **New algorithm**: `guidance/algorithms.md`, skill `/ff-new-algorithm`. `BaseTrainer` owns the epoch loop (`start`), timestep sampling, feedback/advantages, the optimizer step and the velocity KL; only `optimize()` is abstract. Vary behavior through `sampling_context`, `_run_training_step`, `_after_gradient_step` and `_after_optimizer_step` rather than by restating the loop. An algorithm that trains several model copies declares them in `_declare_model_variants()` (`topics/component_variants.md`).
+- **New algorithm**: `guidance/algorithms.md`, skill `/ff-new-algorithm`. `BaseTrainer` owns the cycle loop, acquisition drivers, feedback/advantages, optimizer step, and velocity KL. Generation trainers implement `optimize(samples)`; dataset trainers implement `optimize_batch(batch)`. An algorithm that trains several model copies declares them in `_declare_model_variants()` (`topics/component_variants.md`).
 - **New accelerator**: subclass `acceleration/abc.py::BaseAccelerator` (declare `safety`/`stage`), register in `acceleration/registry.py`
 
 ---
@@ -176,12 +164,28 @@ Timesteps are `[0, 1000]` (scheduler scale); sigmas are `[0, 1]` (flow-matching
 
 ### Adapter Pattern (Models)
 Each model adapter wraps a diffusers pipeline into the `BaseAdapter` interface:
-- `preprocess_func()` — offline encoding (Stage 1)
+- `preprocess_func()` — prompt/input-condition preprocessing and cache projection
+- `pipeline_io_contract` — model-neutral input/output modality and geometry declaration
+- `effective_pipeline_io_contract` — checkpoint-realized specialization of the class contract
+- `prepare_condition_state()` — one validated input-owned runtime realization reused across
+  candidate encoding and model forwards
+- `encode_output_state()` — validated on-the-fly offline target encoding through an optional codec
 - `inference()` — full denoising loop (Stage 3)
 - `forward()` — single-step denoising (Stage 6)
 
 **Per-modality encoders** (`encode_prompt`, `encode_image`, `encode_video`, `encode_audio`) are no-op by default on `BaseAdapter` — override only the modalities your model consumes. `preprocess_func` dispatches to all four and skips any that return `None`, so text/image/video-only adapters need no stub overrides for unused modalities.
 
+Offline condition preparers and codecs declare logical required components without materializing
+them. Condition/output encoders share role-neutral transforms where possible, while callers retain
+explicit official posterior `sample` versus `argmax` semantics. Candidate-specific output context
+cannot overwrite cached or prepared input fields. A separate flow-matching objective reducer lets
+multi-modal SFT/DPO specialize loss aggregation without changing online trajectory reductions.
+
+Input contracts may declare semantic media slots and aggregate cross-type cardinality rules. In
+strict V2 data, an explicit input-only `slot` reserves its argument; unslotted media fills remaining
+slots in declaration order. Outputs reject slots. This keeps algorithm data model-neutral while the
+adapter owns bindings such as first/last frame and ordered heterogeneous references.
+
 **Flat hierarchy**: All adapters inherit directly from `BaseAdapter` — never from another adapter (see constraint #12). Shared logic within a model family uses helper functions, code duplication, or mixins — not adapter subclassing.
 
 Details: `topics/adapter_conventions.md`
@@ -286,19 +290,19 @@ Details: `topics/component_variants.md`.
 
 ### Reward Processing
 `RewardProcessor` dispatches by model type:
-- **Pointwise**: batch by `batch_size`
+- **Pointwise**: applicable sub-batches of at most `batch_size`
 - **Groupwise**: group by `unique_id` (local or distributed path)
 - **Multi-reward**: weighted aggregation
 - **Async**: optional non-blocking computation
 
 ### Advantage Computation
-`AdvantageProcessor` (`advantage/advantage_processor.py`): communication-aware, auto-selects gather vs local path. Strategies: `"sum"` (GRPO) and `"gdpo"`. All reward-based trainers delegate to `self.advantage_processor.compute_advantages()`; the distillation trainer `diffusion-opd` is the exception (its `prepare_feedback()` is a no-op — no reward/advantage stage).
+`AdvantageProcessor` (`advantage/advantage_processor.py`): communication-aware, auto-selects gather vs local path. Strategies: `"sum"` (GRPO) and `"gdpo"`. Runtime-reward trainers delegate to `self.advantage_processor.compute_advantages()`. Feedback-`none` trainers (`diffusion-opd`, DMD2, and TDM) bypass reward and advantage stages structurally.
 
 ### Configuration Hierarchy
 ```
 Arguments (top-level)
 ├── ModelArguments        # model_type, model_path, finetune_type, LoRA config
-├── TrainingArguments     # Algorithm-specific (GRPO/DPO/NFT/AWM subclass)
+├── TrainingArguments     # Algorithm-specific (SFT/offline-DPO/GRPO/DPO/etc.)
 ├── SchedulerArguments    # dynamics_type, timestep_range, num_inference_steps
 ├── DataArguments         # dataset, preprocessing, resolution, sampler_type
 ├── MultiRewardArguments  # reward_model configs (list of RewardArguments)