OOM Error with Stable Diffusion 3-medium and Torch2CoreML on macOS Mac mini M1 8GB RAM

[cekim7]

• Description:
  I'm encountering an out-of-memory error when converting Stable Diffusion 3-medium to CoreML using Torch2CoreML, even with a small latent size.

• Steps to Reproduce:

1. Followed installation guide:
   conda create -n diffusionkit python=3.11 -y
   conda activate diffusionkit
   cd /path/to/diffusionkit/repo
   pip install -e .

huggingface-cli login --token MY_TOKEN

(accepted StabilityAI license terms, and allowed gated access to HuggingFace token.)

2. Run the command: python -m python.src.diffusionkit.tests.torch2coreml.test_mmdit --sd3-ckpt-path stabilityai/stable-diffusion-3-medium --model-version 2b -o for_CoreML_mlpackage --latent-size 16

• Expected Behavior:
  The script should successfully convert the Stable Diffusion model to CoreML.

• Actual Behavior:
  The script fails with the following error message:

RuntimeError: MPS backend out of memory (MPS allocated: 8.08 GB, other allocations: 8.35 GB, max allowed: 9.07 GB). Tried to allocate 12.00 KB on private pool. Use PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 to disable upper limit for memory allocations (may cause system failure).

• System Information:

• macOS Version: 14.0

• Mac Model: arm64, Mac mini M1, 2020

• RAM Size: 8 GB

• Python Version: 3.11.10 (main, Oct 3 2024, 02:26:51) [Clang 14.0.6 ]

• PyTorch Version: 2.5.1

• Additional Information:
  Tried reducing latent-size, but the issue persists.

• Date and Time:
  2024-11-12 00:07:02

[stncttr908]

Got around this on my M4 Mac mini 16 GB by doing the following from the GitHub project root folder:

nano python/src/diffusionkit/__init__.py

import os

os.environ["TOKENIZERS_PARALLELISM"] = "false"
os.environ["PYTORCH_MPS_HIGH_WATERMARK_RATIO"] = "0.0" # Add this line

Ctrl+X to edit and save, rerun and cross your fingers it doesn't crash your poor 8 GB Mac 👍