[trainer,data,model] feat: add SFT and offline DPO

[Jayce-Ping]

Important

This is a stacked PR on top of #219 (feat/precision-aware-model-loading).
Review and merge #219 first; this PR intentionally targets that feature branch rather than main.

Summary

This PR extends Flow-Factory with finite-data SFT and offline DPO while preserving the existing
online RL and distillation paths. The final architecture keeps acquisition, dataset schema,
pipeline semantics, model-specific condition/output transforms, and algorithm objectives as
separate contracts.

• add SFT and offline-DPO trainers with standard finite-dataloader epoch semantics;
• use PyTorch's official DistributedSampler for offline training;
• define one offline epoch as one complete rank-local dataloader traversal;
• keep the public V2 dataset discriminator as type and add an optional input-only semantic
  slot for bindings such as first/last frame;
• cache prompt and input-condition preprocessing only;
• decode and VAE-encode target/chosen/rejected media on the fly, with no target latent cache;
• prepare each stochastic or geometry-bound input condition once and reuse the same realization
  across SFT target binding and all offline-DPO chosen/rejected policy/reference forwards;
• add checkpoint-realized pipeline contracts to exact-resume and condition-cache identity;
• support exact structured audiovisual flow-matching objectives without coupling trainers to
  model-specific layouts; and
• remove the unsupported Wan2 V2V adapter, registry entry, example, and documentation.

Public SFT and offline-DPO smoke datasets

Two independent public, ungated CC0 mini datasets make the offline rows reproducible without
checking media assets into this repository:

• Flow-Factory-SFT-Smoke,
  pinned at 50e7f0e897ed2ce4160b14340fc8a982fe6c6919;
• Flow-Factory-Offline-DPO-Smoke,
  pinned at b6ec289238eefdd6c855b7fb8f87ccc80ee3040f.

Each repository contains 32 deterministic records for each of the 10 main semantic modes plus the
supplemental image-i2i contract gate. The checked-in builder reconstructs all procedural media,
the publisher validates the complete staging tree before any Hub mutation, and the preparer
materializes exactly two rank-local batches from immutable revisions. Input conditions and outputs
reuse the official ordered PipelineIOContract; current assets cover image, video, and ordered
(video, audio) outputs, while the schema remains compatible with a future audio-only adapter.
No target/chosen/rejected VAE latent is cached.

The final remote revisions are public and non-gated; sampled manifests are byte-identical to the
deterministic build and exercise the released diffusers==0.40.0 contract.

Model and workflow coverage

Wan

• T2V target-video encoding for Wan 2.1/2.2 checkpoints.
• I2V realizes a checkpoint-specific endpoint contract: ordinary and expanded-timestep models
  accept exactly first_frame, dedicated FLF2V requires both endpoints, and Wan2.2 I2V-A14B also
  supports an optional VAE-only last_frame.
• Endpoint conditions are encoded once with the official posterior mode and reused by both
  offline-DPO arms.
• Condition regions remain pinned through forward noising and loss masks.

LTX2

• T2AV and I2AV exact ordered (video, audio) target codecs, including LTX 2.3.
• Official video/audio clocks, log-mel conversion, latent packing, per-component times, and joint
  objective reduction.
• I2AV replaces and pins the first target latent frame with the prepared condition posterior mode
  and excludes it from the active loss region.
• Offline velocity requests use the exact raw model velocity; online rollout keeps the official
  generation round trip.

MiniMax H3

• T2VA, sparse FL2VA, and ordered heterogeneous Ref2VA expose an exact ordered
  (video, audio) output contract.
• FL2VA supports first_frame, last_frame, or both; last-only input is not conflated with a
  first-frame positional shorthand.
• Ref2VA supports 1-12 globally ordered image/video/audio references and requires at least one
  image or video.
• Official condition-prefix randomness is realized once per input batch and shared by all offline
  candidates and model scopes.
• Video/audio target modes, packing, geometry, clocks, component-specific noising, and H3's
  data-ward velocity convention remain adapter-owned.

Real-weight GPU validation completed

The complete dynamic smoke scope retains successful terminal evidence for 144/144 unique jobs:

Scope	Result	Coverage
Main matrix	120/120	10 semantic modes × 3 distributed backends × 4 algorithms
Formal variant gates	22/22	Wan 2.1/2.2 routing variants, LTX 2.3, and H3 FL2VA dual-endpoint coverage
Supplemental	2/2	Flux1-Kontext image-to-image SFT and offline DPO

The main matrix used DDP, DeepSpeed ZeRO-2, and FSDP2 with GRPO, SFT, offline DPO, and TDM.
Every semantic mode completed all 12 backend/algorithm cells:

Mode ID	Pipeline/workflow	Representative checkpoint	Result
sd35-t2i	SD3.5 text-to-image	stabilityai/stable-diffusion-3.5-medium	12/12
bagel-mri2i	Bagel ordered two-reference-images-to-image	ByteDance-Seed/BAGEL-7B-MoT	12/12
wan-t2v	Wan text-to-video	Wan-AI/Wan2.1-T2V-1.3B-Diffusers	12/12
wan-i2v-first	Wan first-frame-to-video	Wan-AI/Wan2.2-TI2V-5B-Diffusers	12/12
wan-flf2v	Wan first/last-frame-to-video	Wan-AI/Wan2.1-FLF2V-14B-720P-diffusers	12/12
ltx2-t2av	LTX2 text-to-audio-video	Lightricks/LTX-2	12/12
ltx2-i2av	LTX2 image-to-audio-video	Lightricks/LTX-2	12/12
h3-t2va	MiniMax H3 text-to-video-audio	MiniMaxAI/MiniMax-H3	12/12
h3-fl2va	MiniMax H3 sparse first/last-frame-to-video-audio	MiniMaxAI/MiniMax-H3	12/12
h3-ref2va	MiniMax H3 ordered-reference-to-video-audio	MiniMaxAI/MiniMax-H3	12/12

All 144 jobs retain successful multi-rank, real-weight training evidence and completed the intended
smoke length: two acquisition/training cycles for GRPO/TDM or two rank-local finite-loader batches
for SFT/offline DPO. Enhanced finite-loss/gradient, component-update, and route evidence was
captured for the instrumented subset, including all six strict A14B gates. Some earlier direct-run
logs predate the full command/environment/metrics artifact contract; missing historical fields
remain disclosed as gaps and were not backfilled.

Variant and optimizer gates

• 22/22 dynamic variant-gate jobs: Wan T2V/TI2V family (6), Wan first-frame I2V (6), Wan
  endpoint routing (4, including two corrected redundant executions of the dedicated FLF2V
  checkpoint), dg845/LTX-2.3-Diffusers T2AV (2) and I2AV (2), and H3 FL2VA
  first-plus-last (2).
• Six unique strict Wan A14B jobs passed explicit dual-transformer routing checks, including
  both sides of the timestep boundary.
• 132/132 static checkpoint-variant cells passed: 11 variants × 3 backends × 4 algorithms.
• 4/4 positive Muon jobs passed: DDP and FSDP2 SFT, plus DDP and FSDP2 mixed-role TDM.
• The expected DeepSpeed ZeRO-2 Muon negative gate rejected the unsupported plan before model
  loading.

These are reduced-geometry, finite-length execution smokes. They establish archived real-weight
execution coverage across loading, distributed backends, model-specific conditioning, structured
image/video/audio-video paths, optimizers, and finite-data termination; the instrumented strict
gates provide deeper routing/update evidence where required. They do not claim that every
historical log contains the full formal artifact set, nor do they claim long-run convergence,
reward improvement, visual/audio quality parity, or numerical parity.

The executable matrix contract and acceptance criteria are documented in
guidance/gpu_validation.md.

Final merge review

• Full repository test suite: 2563 passed, 18 skipped.
• All 88 checked-in example configurations load successfully.
• Public documentation, agent knowledge, source docstrings, examples, and all six existing agent
  skills were reconciled with the current execution/runtime contracts.
• Registered Wan, LTX2, and MiniMax H3 workflows resolve with GRPO/SFT/offline-DPO/TDM under the
  pinned Diffusers contract.
• All 6/6 agent skills pass schema validation; all 74 local agent-document links and
  anchors resolve. Changed-file formatting/import checks, compilation, and diff hygiene passed.

Review notes

• DeepSpeed ZeRO-3 remains unsupported framework-wide and is outside the matrix.
• Bagel FSDP2 uses a globally uniform two-reference layout in the main matrix; ragged per-rank
  reference rounds remain a separate fail-fast stress case.
• TDM with DeepSpeed uses AdamW roles; FSDP2 multi-role validation requires use_orig_params=True.
• Capacity or infrastructure failures are classified separately from model-support failures.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.