PowerImaginator integrates 2D image editing models with an efficient 3D reconstruction (VGGT) and novel view synthesis (BlendPCR) pipeline, enabling users to harness the vast world knowledge and precise controllability of image models while creating open-world 3D environments.
This repository is experimental, and setting it up requires unusual technical knowledge. Your experience will probably be best when exploring this codebase in combination with an agentic AI harness such as OpenCode. AI agents can help with system configuration, installation, and startup, in addition to adding new features and resolving bugs.
Running the full PowerImaginator stack requires a modern GPU compute platform (NVIDIA CUDA, AMD ROCm, or Apple Silicon) and consumes significant memory, especially VRAM. If you do not have access to a compatible GPU, or you encounter out-of-memory errors, we recommend using a service such as Runpod or Vast.ai to rent GPU instances on-demand. Depending on availability and pricing, you may find it cheaper to split ComfyUI and the PowerImaginator backend across two separate instances, or edit the backend/main.py file to run VGGT exclusively on CPU (this is fully functional, and is the default on AMD or Apple Silicon systems, but it is slower than CUDA).
PowerImaginator is written in Svelte and Python. It depends on ComfyUI for 2D image generation, and VGGT for lifting 2D images to 3D scenes. Optionally, LLM chat is provided via llama-server.
Before proceeding, ensure you have installed Git, Python, and Node.js.
Clone and open ComfyUI via Git:
git clone --recursive https://github.com/Comfy-Org/ComfyUI.git
cd ComfyUI
python -m venv .venv
source .venv/bin/activateFollow the ComfyUI manual installation instructions if you have an NVIDIA, AMD, or Intel GPU. Otherwise, simply run:
pip install -r requirements.txt
cd ..Once you've installed ComfyUI, you need to download models. We recommend Flux.2 Klein 4B Distilled as a lightweight baseline compatible with many systems, including Apple Silicon devices. If resources permit, we also recommend Qwen-Image-Edit-2511. The ComfyUI community has created many workflows covering a wide range of use cases and hardware configurations. If you're new to 2D image generation and editing, we recommend familiarizing yourself with the basics of ComfyUI usage on your device before setting up PowerImaginator.
If you'd like to use the LLM chat features, follow the instructions in the llama.cpp readme to install llama-server and download models. Like image generation models, many LLM variants are available to suit a wide variety of hardware. This guide does not further detail llama-server configuration or usage. Web search and AI agents will help here. LLM integration is not essential, and we recommend starting without it.
Open Terminal, then clone and open the PowerImaginator Git repo recursively including submodules, as shown below:
git clone --recursive https://github.com/PowerImaginator/powerimaginator2.git
cd powerimaginator2Install frontend dependencies with npm:
cd frontend
npm i -D
cd ..Next, install backend dependencies with pip:
cd backend
python -m venv .venv
source .venv/bin/activate
pip install -r vggt/requirements.txt
pip install "fastapi[standard]"
cd ..By default, ComfyUI launches on port 8188, the PowerImaginator VGGT backend launches on port 8189, and the frontend launches on port 5173. You will need to make these ports externally accessible if using Runpod or another remote provider.
cd ComfyUI
source .venv/bin/activate
python main.py --enable-cors-headerNote: --enable-cors-header is essential for allowing PowerImaginator to communicate with the external ComfyUI server.
PowerImaginator will scan your ComfyUI workflows and display compatible workflows in its interface. PowerImaginator sends reference images and prompts to ComfyUI, and ComfyUI sends generated outputs back to PowerImaginator. Following is a tutorial for setting up your first PowerImaginator-compatible workflow:
- Open ComfyUI and load the "Flux.2 [Klein] 4B Distilled: Image Edit" template workflow
- Rename the first "Load Image" node
##PI_IN1(double-click in the title bar of the node to rename) - Create a "Get Image Size" node. Connect the output of the
##PI_IN1node to "Get Image Size" - Create an "Empty Image" node. Connect the
width,height, andbatch_sizeoutputs of "Get Image Size" to the corresponding inputs of "Empty Image" - In the
colorfield of "Empty Image", type either0xff00ff(for magenta) or0x00ff00(for green), then push the Enter key. These will become special colors which will fill the transparent area of the image, like a green screen. This enables writing in your prompt "replace the magenta area with ..." or "replace the green screen with ..." You may test other colors, but pure magenta and pure green are common special colors in image processing. - Create an "Image Composite Masked node"
- Connect the
IMAGEoutput of##PI_IN1to thedestinationinput of "Image Composite Masked" (it should also remain connected to "Get Image Size") - Connect the
MASKoutput of##PI_IN1to themaskinput of "Image Composite Masked" - Connect the
IMAGEoutput of "Empty Image" to thesourceinput of "Image Composite Masked" - Connect the
IMAGEoutput of "Image Composite Masked" to theimageinput of the "Image Edit (Flux.2 Klein 4B Distilled) node. - Replace the content of the
promptfield in the "Image Edit (Flux.2 Klein 4B Distilled)" subgraph node with##PI_PROMPT - Rename the "Save Image" node to
##PI_OUT1 - Press Ctrl+S to name and save your workflow
After creating a new ComfyUI workflow, click the Settings icon in the PowerImaginator sidebar, then click the button labeled "Refresh Workflows." Otherwise, your workflow will not appear inside PowerImaginator.
cd powerimaginator2/backend
source .venv/bin/activate
python main.pyWhen you create your first scene with PowerImaginator, the backend will attempt to fetch the facebook/VGGT-1B-Commercial checkpoint from HuggingFace. If you're not planning to use PowerImaginator commercially, you may edit main.py to replace VGGT-1B-Commercial with VGGT-1B. This avoids the authentication requirement.
Either way, depending on your network connection speed, you may experience a delay of several minutes to an hour when you first click "Apply to Scene" in the PowerImaginator UI. Subsequent runs will always be much faster.
cd powerimaginator2/frontend
npm run devYou're now ready to open the PowerImaginator application, available at http://localhost:5173
When you first launch PowerImaginator, you'll see an empty canvas with a checkerboard background, and a sidebar on the left with a plus icon and a settings icon. If you need to connect PowerImaginator to a remote server such as a Runpod instance, click the settings icon, and configure the ComfyUI, VGGT, and (optionally) llama.cpp URLs to the correct addresses. If you're running the entire stack on your own computer, the default localhost values should work out-of-the-box.
Underneath the ComfyUI URL and "Refresh Workflows" button, a status line indicates the number of workflows discovered on your ComfyUI instance, and the number of valid PowerImaginator workflows.
If you see zero discovered workflows: you may have no saved workflows in ComfyUI, or the ComfyUI URL is inaccessible. To test the ComfyUI URL, click "Refresh Workflows." If an error dialog appears, the URL is either incorrect, or the server is not reachable over the network. Double-check that you've started ComfyUI with --enable-cors-header, and that the URL points to the specific port or proxy for your ComfyUI instance.
If some workflows are discovered, but there are zero valid workflows, then all your workflows are missing ##PI_OUT1 "Save Image" or "Preview Image" nodes. Refer to the ComfyUI Workflow Setup section earlier in this guide.
If, once you start creating content, you click "Apply to Scene" and encounter a network error, this means the VGGT URL may be incorrect, the VGGT instance may be inaccessible, or the backend encountered an error.
To begin, click the plus icon in the sidebar.
By default, this creates an "Upload Only" image collection. Clicking the "Add Output" button creates an image slot. If you click on this slot, the image editor appears to the right of the sidebar. If you want to kickstart your scene with an existing image, click the "Upload" button in the top bar of the editor, upload an image, then click the "Apply to Scene" button underneath the list of image slots in the sidebar. A "Rebuilding..." status indicator will appear above the viewport. After a few moments, the checkerboard viewport on the rightmost side of the screen will display similar content to the image you uploaded. Click inside the viewport, and you can use traditional fly camera WASD and mouse controls to navigate the 3D scene.
As you navigate the 3D scene, you'll notice empty regions, outside the view range of the original image. Click the plus icon again in the sidebar, then click the "Upload Only" dropdown to reveal compatible ComfyUI workflows. Select your ComfyUI workflow from this list.
Your workflow should have at least one image slot in the "Context" section. Click this image slot to open the image editor, then click "Capture Viewport" in the top bar of the editor. You'll see the content of the viewport copied into the context image.
Next, write a prompt in the "Prompt" field. The prompt should clearly reference the background color you chose in the earlier ComfyUI Workflow Setup section, and should specify what content you want to add in this empty area. You may also need to prompt for blur removal if you've zoomed in closer to any surface. Prompt writing is an iterative process, and different combinations of prompts and images will yield vastly different results. An example prompt is as follows:
Remove the magenta color and fill it with new content. Completely remove motion blur. Restore the image using high-resolution photorealistic textures. The scene is a [description of the scene you want to generate]
Now click the "Generate" button underneath the prompt field. When generation finishes, you should see content appear in the output image slot. Click the output image slot to open the image editor for closer inspection. If the image is satisfactory, click the "Apply to Scene" button to stitch this new image into the existing scene. Otherwise, you may want to adjust the prompt, or moving the viewport camera and re-capture the viewport in the context image slot.
You'll notice a variety of other editing tools available in the image editor. For output image slots, the "Mask" operation with the "Box" tool is particularly effective for removing fine-grained geometry such as lamp posts, which tends to confuse the 3D reconstruction model. Copying and pasting are also helpful for feeding output images back into input image slots for edit loops, or transferring images between PowerImaginator and other applications.
PowerImaginator is an early-stage research prototype, and remains challenging to use for producing high-fidelity scenes. We hope our community will kindly share their experiences and advice with us (and each other!) via our Discord server. Furthermore, we highly encourage use of AI agents to "vibe code" your own customizations to PowerImaginator, and gain a greater understanding of how it works internally.
If you're interested to delve deeper, we're hoping for help with the following:
- Train an efficient and widely deployable generative novel view synthesis model, preferably starting from an existing image edit base model, then finetuning for use in a 3D generation harness such as PowerImaginator. dx8152's Qwen Image Edit 2511 Gaussian Splash LoRA is a strong step in this direction.
- If you're interested to help with this, please reach out via Discord. We may be able to provide advice and assistance.
- vggt-dataset-builder was an attempt at building a dataset for this task, though it uses an older renderer not matching the current PowerImaginator pipeline.
- Investigate feed-forward Gaussian splatting and other 3D reconstruction methods instead of VGGT. AnySplat and ReSplat are two promising approaches, though both require multiple input images, along with a secondary optimization pass after their initial feed-forward prediction, to achieve high quality results.