Repository navigation
PNDMScheduler Does Not Compatible with DDPMPipeline #7354
Description
Describe the bug
I was trying to test different schedulers under DDPMPipeline. And an error occurred if I use PNDMScheduler beforehand I have found that PNDMScheduler should be compatible with DDPMPipeline following the official tutorial.
pipeline = DDPMPipeline(unet=model, scheduler=noise_scheduler)
pipeline.scheduler.compatiblesAnd the output is this:
[diffusers.schedulers.scheduling_ddim.DDIMScheduler,
diffusers.schedulers.scheduling_euler_discrete.EulerDiscreteScheduler,
diffusers.schedulers.scheduling_dpmsolver_multistep.DPMSolverMultistepScheduler,
diffusers.schedulers.scheduling_lms_discrete.LMSDiscreteScheduler,
diffusers.schedulers.scheduling_dpmsolver_singlestep.DPMSolverSinglestepScheduler,
diffusers.schedulers.scheduling_deis_multistep.DEISMultistepScheduler,
diffusers.utils.dummy_torch_and_torchsde_objects.DPMSolverSDEScheduler,
diffusers.schedulers.scheduling_k_dpm_2_discrete.KDPM2DiscreteScheduler,
diffusers.schedulers.scheduling_k_dpm_2_ancestral_discrete.KDPM2AncestralDiscreteScheduler,
diffusers.schedulers.scheduling_unipc_multistep.UniPCMultistepScheduler,
diffusers.schedulers.scheduling_heun_discrete.HeunDiscreteScheduler,
diffusers.schedulers.scheduling_euler_ancestral_discrete.EulerAncestralDiscreteScheduler,
diffusers.schedulers.scheduling_pndm.PNDMScheduler,
diffusers.schedulers.scheduling_ddpm.DDPMScheduler]
And the main error massage is that
File [c:\Users\User\miniconda3\envs\Pytorch\lib\site-packages\diffusers\pipelines\ddpm\pipeline_ddpm.py:117](file:///C:/Users/User/miniconda3/envs/Pytorch/lib/site-packages/diffusers/pipelines/ddpm/pipeline_ddpm.py:117), in DDPMPipeline.__call__(self, batch_size, generator, num_inference_steps, output_type, return_dict)
[114](file:///C:/Users/User/miniconda3/envs/Pytorch/lib/site-packages/diffusers/pipelines/ddpm/pipeline_ddpm.py:114) model_output = self.unet(image, t).sample
[116](file:///C:/Users/User/miniconda3/envs/Pytorch/lib/site-packages/diffusers/pipelines/ddpm/pipeline_ddpm.py:116) # 2. compute previous image: x_t -> x_t-1
--> [117](file:///C:/Users/User/miniconda3/envs/Pytorch/lib/site-packages/diffusers/pipelines/ddpm/pipeline_ddpm.py:117) image = self.scheduler.step(model_output, t, image, generator=generator).prev_sample
[119](file:///C:/Users/User/miniconda3/envs/Pytorch/lib/site-packages/diffusers/pipelines/ddpm/pipeline_ddpm.py:119) image = (image / 2 + 0.5).clamp(0, 1)
[120](file:///C:/Users/User/miniconda3/envs/Pytorch/lib/site-packages/diffusers/pipelines/ddpm/pipeline_ddpm.py:120) image = image.cpu().permute(0, 2, 3, 1).numpy()
TypeError: PNDMScheduler.step() got an unexpected keyword argument 'generator'In DDPMPipeline.scheduler.step( ) function, it takes generator as the argument. But in class diffusers.PNDMScheduler step( ) function shown in official doc, the function only takes model_output (torch.FloatTensor), timestep (int), sample (torch.FloatTensor), return_dict (bool).
Moreover, I also find out that HeunDiscreteScheduler has the same problem
Reproduction
from diffusers import DDIMScheduler
from diffusers import UNet2DModel
from diffusers import PNDMScheduler
model = UNet2DModel(
sample_size=config.image_size,
in_channels=1,
out_channels=1,
layers_per_block=2,
block_out_channels=(128,128,256,512),
down_block_types=(
"DownBlock2D",
"DownBlock2D",
"AttnDownBlock2D",
"DownBlock2D",
),
up_block_types=(
"UpBlock2D",
"AttnUpBlock2D",
"UpBlock2D",
"UpBlock2D",
),
)
noise_scheduler = PNDMScheduler()
pipeline = DDPMPipeline(unet=model, scheduler=noise_scheduler)
print(pipeline.scheduler.compatibles)
images = pipeline(
batch_size=1,
num_inference_steps = 50).images Logs
TypeError Traceback (most recent call last)
Input In [58], in <cell line: 26>()
24 pipeline = DDPMPipeline(unet=model, scheduler=noise_scheduler)
25 pipeline.scheduler.compatibles
---> 26 images = pipeline(
27 batch_size=1,
28 num_inference_steps = 50).images
File c:\Users\User\miniconda3\envs\Pytorch\lib\site-packages\torch\utils\_contextlib.py:115, in context_decorator.<locals>.decorate_context(*args, **kwargs)
112 @functools.wraps(func)
113 def decorate_context(*args, **kwargs):
114 with ctx_factory():
--> 115 return func(*args, **kwargs)
File c:\Users\User\miniconda3\envs\Pytorch\lib\site-packages\diffusers\pipelines\ddpm\pipeline_ddpm.py:117, in DDPMPipeline.__call__(self, batch_size, generator, num_inference_steps, output_type, return_dict)
114 model_output = self.unet(image, t).sample
116 # 2. compute previous image: x_t -> x_t-1
--> 117 image = self.scheduler.step(model_output, t, image, generator=generator).prev_sample
119 image = (image / 2 + 0.5).clamp(0, 1)
120 image = image.cpu().permute(0, 2, 3, 1).numpy()
TypeError: PNDMScheduler.step() got an unexpected keyword argument 'generator'System Info
I cannot use this command but I can show some info.
diffusers 0.26.3 pyhd8ed1ab_0 conda-forge
pytorch 2.0.1 py3.10_cuda11.8_cudnn8_0 pytorch
OS: Windows 11
Activity
sayakpaul commentedon Mar 18, 2024
@yiyixuxu could you give this a look?
yiyixuxu commentedon Mar 19, 2024
hey thanks for the issue!
normally we inspect the signature of the scheduler and only pass
generatorif it would be accepted e.g. stable diffusion pipeline here.diffusers/src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py
Line 580 in 80ff4ba
do you want to open a PR to add this method to DDPMPipeline?
KeepNoob commentedon Mar 20, 2024
github-actions commentedon Apr 16, 2024
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.
Please note that issues that do not follow the contributing guidelines are likely to be ignored.
github-actions commentedon May 11, 2024
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.
Please note that issues that do not follow the contributing guidelines are likely to be ignored.
sayakpaul commentedon Jun 30, 2024
I think you opened the PR to the wrong repository. Could you please check? @KeepNoob
github-actions commentedon Oct 9, 2024
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.
Please note that issues that do not follow the contributing guidelines are likely to be ignored.
a-r-r-o-w commentedon Nov 20, 2024
Gentle ping @KeepNoob to open the PR to diffusers:main instead of the main branch of your fork. I think this is still an issue and we would love to have your help addressing this!
github-actions commentedon Dec 14, 2024
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.
Please note that issues that do not follow the contributing guidelines are likely to be ignored.
NaCr05 commentedon Oct 22, 2026
@KeepNoob and maintainers, I revisited this with a small CPU reproduction and would like to coordinate with the existing fork PR before proceeding.
I ran checkout 73faca3; the relevant source/test blobs matched the inspected main snapshot 48cdbae. On Windows CPU with PyTorch 2.13.0 and float32, I used the existing tiny DDPM test configuration: a real 8×8 UNet, eight inference steps, and an unmodified public
DDPMPipeline(...)call.Both
PNDMScheduler(skip_prk_steps=True)andFalsefailed after the first UNet evaluation with:DDPM and DDIM controls returned finite tensors of shape
(1, 3, 8, 8); the two existing DDPM inference baseline tests also passed. These checks did not use a patched scheduler or production model weights.Would a narrowly scoped follow-up that passes
generatoraccording to the scheduler'sstepsignature be welcome? I would propose real tiny-pipeline regression cases for both PNDM settings, checking shape and finiteness while retaining DDPM/DDIM controls. This would address the PNDM argument contract only: broader scheduler compatibility would need separate consideration of input scaling and initial noise sigma.@KeepNoob, are you still planning to submit the original change upstream, or would help continuing it be useful? I would appreciate maintainer confirmation of the scope before implementation.
These local checks and this write-up were AI-assisted; no fix has been implemented.