Skip to content

AutoencoderRAE loading error with older transformers #13279

Open

Description

@arijit-hub

Describe the bug

Hye,

I am trying to use AutoencoderRAE, unfortunately I cannot load it and it seems to break by giving the following error:

File ".venv/lib/python3.13/site-packages/transformers/models/dinov2_with_registers/modeling_dinov2_with_registers.py", line 529, in _init_weights
    ).to(module.weight.dtype)
      ^^
AttributeError: 'NoneType' object has no attribute 'to'

I think its because of older tokenizers, transformers and huggingface-hub versions.

I am in the following versions for them:

transformers==4.55.4
tokenizers==0.21.2
huggingface-hub==0.36.2
diffusers==0.37.0

Updating them to the following fixes the problem:

transformers==5.3.0
tokenizers==0.22.2
huggingface-hub==1.7.1
diffusers==0.37.0

However, it breaks my metric script for calculating ImageReward! Is there a solution for this without updating packages?

Reproduction

The package versions must be:

transformers==4.55.4
tokenizers==0.21.2
huggingface-hub==0.36.2
diffusers==0.37.0

Then the simple loading fails:

from diffusers import AutoencoderRAE

model = AutoencoderRAE.from_pretrained(
    "nyu-visionx/RAE-dinov2-wReg-base-ViTXL-n08"
).to("cuda").eval()

Logs

File ".venv/lib/python3.13/site-packages/transformers/models/dinov2_with_registers/modeling_dinov2_with_registers.py", line 529, in _init_weights
    ).to(module.weight.dtype)
      ^^
AttributeError: 'NoneType' object has no attribute 'to'

System Info

  • 🤗 Diffusers version: 0.37.0
  • Platform: Linux-5.15.0-119-generic-x86_64-with-glibc2.31
  • Running on Google Colab?: No
  • Python version: 3.13.5
  • PyTorch version (GPU?): 2.8.0+cu128 (True)
  • Flax version (CPU?/GPU?/TPU?): not installed (NA)
  • Jax version: not installed
  • JaxLib version: not installed
  • Huggingface_hub version: 0.36.2
  • Transformers version: 4.55.4
  • Accelerate version: 1.13.0
  • PEFT version: not installed
  • Bitsandbytes version: not installed
  • Safetensors version: 0.7.0
  • xFormers version: not installed
  • Accelerator: NVIDIA RTX A6000, 49140 MiB
    NVIDIA RTX A6000, 49140 MiB
  • Using GPU in script?: Yes
  • Using distributed or parallel set-up in script?: No

Who can help?

@sayakpaul @kashif

Activity

  1. kashif commented on Mar 17, 2026

    @kashif
    Contributor

    thanks @arijit-hub checking

  2. self-assigned this
    on Mar 17, 2026
  3. kashif commented on Mar 18, 2026

    @kashif
    Contributor

    I have fixed ImageReward to work with transformers v5, can you try with:

      pip install "ImageReward @ git+https://github.com/kashif/ImageReward.git@transofrmers-v5"
    
  4. arijit-hub commented on Mar 18, 2026

    @arijit-hub
    Author

    Hye,

    Thanks for looking into it so quickly. Really appreciate it.

    Unfortunately, this leads to import error in ImageReward due to transformers :(

    ImportError: cannot import name 'apply_chunking_to_forward' from 'transformers.modeling_utils'
    

    Maybe doing something in diffusers makes more sense than changing ImageReward?

    Again, thanks a lot for looking into it!

  5. lavinal712 commented on Oct 4, 2026

    @lavinal712
    Contributor

    small patch to src/diffusers/models/modeling_utils.py

    @contextmanager
    def no_init_weights():
        """
        Context manager to globally disable weight initialization to speed up loading large models. To do that, all the
        torch.nn.init function are all replaced with skip.
        """
    
        def _skip_init(*args, **kwargs):
            pass
    
        hf_modeling_utils = None
        hf_init_weights = None
        if is_transformers_available():
            from transformers import modeling_utils as hf_modeling_utils
    
            hf_init_weights = hf_modeling_utils._init_weights
            hf_modeling_utils._init_weights = False
        for name, init_func in TORCH_INIT_FUNCTIONS.items():
            setattr(torch.nn.init, name, _skip_init)
        try:
            yield
        finally:
            # Restore the original initialization functions
            for name, init_func in TORCH_INIT_FUNCTIONS.items():
                setattr(torch.nn.init, name, init_func)
            if hf_modeling_utils is not None:
                hf_modeling_utils._init_weights = hf_init_weights
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions