Add precision-aware model loading and MiniMax H3 training - #219
Open
Jayce-Ping wants to merge 26 commits into
Open
Add precision-aware model loading and MiniMax H3 training#219Jayce-Ping wants to merge 26 commits into
Jayce-Ping wants to merge 26 commits into
Conversation
Let DMD2, TDM, and TDM-R1 preserve adapter-specific inference parameters through extra_kwargs while continuing to reject known retired algorithm controls.
Make video frame geometry explicit across training algorithms, remove DMD2's custom argument rejection path, and require the released diffusers 0.40 MiniMax H3 API.
Construct MiniMaxH3Blocks explicitly so diffusers 0.40.0 can load local modular checkpoints without its unresolved default-blocks path.
Bypass diffusers 0.40.0's broken local from_pretrained blocks forwarding while retaining its public constructor and component-spec loading behavior.
Use the official workflow-pruned from_pretrained path unchanged for both Hugging Face repo IDs and local unified checkpoint directories, removing redundant spec mutation and partition validation.
Keep the official ModularPipeline loader for both Hub IDs and local directories, rebinding only local ComponentSpec sources so component loading stays offline.
Convert the B=1 empty decoded-video batch to a per-sample None before constructing structured H3 samples for TDM and DMD2 rollouts.
Avoid overriding Diffusers-managed FP32 islands in the validated T2VA recipes.
Separate component load dtype from storage precision and centralize target, auxiliary, and reward loading so distributed backends cannot mis-handle aliases or replicated weights.
Prevent target-owned roots from re-entering auxiliary device management, make FSDP validation collective-safe, and remove unused loading-plan metadata.
Keep prepared logical targets in place while moving sibling modules needed by composite pseudo-pipelines such as Bagel.
Align runtime documentation with Bagel's partial-root residency behavior and guard fully excluded target roots from movement.
Point both SenseNova training examples at the intended SFT initialization.
Enable velocity-based KL regularization and use 14 denoising steps consistently in SenseNova examples.
Keep only the SFT model path update while deferring experimental KL and sampling settings until validated.
Use the shared vid_prompt dataset and align general GRPO, reward, EMA, and optimizer settings with the LTX2 T2AV recipe while preserving H3-specific geometry and guidance constraints.
Keep preprocessing at B=1 because MiniMax H3 does not accept batched outer workflow inputs.
Reuse the batched media-free decode contract for Bagel, map reference guidance through adapters, and accept integral scheduler coordinates so TDM can replay native trajectories without model-specific patches.
Flatten trajectory boundaries into ordinary backend accumulation work items so each graph is released after backward while preserving rollout-level feedback and optimizer-step semantics.
Allow backward-compatible full or selective block policies, keep checkpoint ownership singular under FSDP, and expose an ordered MiniMax H3 checkpoint-unit contract for memory/performance tuning.
Decouple evaluation sampling and media logging from optional reward computation so reward-free distillation runs still publish generated outputs.
Publish the working 512x768x124 configuration with sanitized save paths, noting that training needs over 200 steps to approach convergence.
Jayce-Ping
force-pushed
the
feat/precision-aware-model-loading
branch
from
August 31, 2026 00:42
2d5c477 to
9c8331d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Test plan
python -m pytest -q— 1722 passed, 14 skipped