DDO is a model-agnostic offload manager for low-VRAM Diffusers-style inference. It provides dense-weight routes and official Diffusers compatibility routes while preserving quantized backend math.
Follow the installation guide, then choose full-pipeline, staged or Modular Diffusers integration.
Normal linear use requires no native allocator or compiler. Native VMM is optional:
it uses a matching packaged DLL/.so and bootstrap before CUDA initialization.
Experimental version 0.1.0rc3 is available on PyPI. The installation guide
uses published packages, without requiring a source clone or local wheel files.
from diffusers_dynamic_offloader import enable_offload
result = enable_offload(transformer, preset="auto", component="transformer")
transformer = result.moduleThis snippet selects the default linear route. See native VMM integration for its separate loading contract.
Read the documentation website or browse the Markdown guides below.
- User guides: installation, integration, presets and reporting.
- Developer guides: architecture, diagnostics, builds and validation.
Model runners, custom-block integration and measured comparisons live in DDO examples. See the test hardware and support boundaries when interpreting results. Selected workloads do not establish universal speedups or model support.
Apache-2.0; preserve required copyright, license and attribution notices.