How can I use only the images in the video without any prompts? TypeError: 'numpy.float64' object is not iterable #52
Description
{
"batch_name":"Example_DGSpitzer",
"n_batch": 1,
"prompts":[],
"width":576,
"height":832,
"bit_depth_output":8,
"seed":3521683325,
"seed_behavior":"iter",
"sampler":"dpmpp_2s_a",
"steps":50,
"scale":7,
"ddim_eta":0.0,
"filename_format":"{timestring}{index}{prompt}.png",
"use_init":true,
"init_image":"https://cdn.pixabay.com/photo/2022/07/30/13/10/green-longhorn-beetle-7353749_1280.jpg",
"strength":0.0,
"use_mask":true,
"use_alpha_as_mask":false,
"invert_mask":false,
"mask_file":"https://www.filterforge.com/wiki/images/archive/b/b7/20080927223728%21Polygonal_gradient_thumb.jpg",
"mask_brightness_adjust":1.0,
"mask_contrast_adjust":1.0,
"overlay_mask":true,
"mask_overlay_blur":5,
"mean_scale":0,
"var_scale":0,
"exposure_scale":1,
"exposure_target":0.5,
"colormatch_scale":1,
"colormatch_image":"https://www.saasdesign.io/wp-content/uploads/2021/02/palette-3-min-980x588.png",
"colormatch_n_colors":4,
"ignore_sat_weight":0,
"clip_name":"ViT-L/14",
"clip_scale":1,
"aesthetics_scale":0,
"cutn":1,
"cut_pow":0.0001,
"init_mse_scale":0,
"init_mse_image":"https://cdn.pixabay.com/photo/2022/07/30/13/10/green-longhorn-beetle-7353749_1280.jpg",
"blue_scale":1,
"gradient_wrt":"x0_pred",
"gradient_add_to":"both",
"decode_method":"linear",
"grad_threshold_type":"dynamic",
"clamp_grad_threshold":0.2,
"clamp_start":0.2,
"clamp_stop":0.01,
"cond_uncond_sync":true,
"animation_prompts":{
},
"animation_mode":"Video Input",
"max_frames":100,
"diffusion_cadence":"2",
"border":"warp",
"angle":"0:(0)",
"zoom":"0:(1.04)",
"translation_x":"0:(5sin(23.14*t/10))",
"translation_y":"0:(0.5)",
"translation_z":"0:(0.5)",
"rotation_3d_x":"0:(0)",
"rotation_3d_y":"0:(0)",
"rotation_3d_z":"0:(0)",
"flip_2d_perspective":false,
"perspective_flip_theta":"0:(0)",
"perspective_flip_phi":"0:(t%15)",
"perspective_flip_gamma":"0:(0)",
"perspective_flip_fv":"0:(53)",
"noise_schedule":"0:(0.02)",
"strength_schedule":"0:(0.55),49:(0.1),51:(0.65)",
"contrast_schedule":"0:(1.0)",
"color_coherence":"Match Frame 0 LAB",
"use_depth_warping":true,
"midas_weight":0.3,
"near_plane":200,
"far_plane":10000,
"fov":40,
"padding_mode":"border",
"sampling_mode":"bicubic",
"save_depth_maps":false,
"video_init_path":"./input/2.mp4",
"extract_nth_frame":1,
"overwrite_extracted_frames":true,
"use_mask_video":false,
"video_mask_path":"",
"interpolate_key_frames":false,
"interpolate_x_frames":4,
"resume_from_timestring":false,
"resume_timestring":"20220829210106",
"fps":24
}
(dsd) E:\tools\DeforumStableDiffusionLocal-main>python run.py --enable_animation_mode --settings "./examples/runSettings_AnimationExtra.txt"
models_path: E:\tools\DeforumStableDiffusionLocal-main\models
output_path: E:\tools\DeforumStableDiffusionLocal-main\output
configs\v1-inference.yaml exists
config_path: E:\tools\DeforumStableDiffusionLocal-main\configs\v1-inference.yaml
ckpt_path: E:\tools\DeforumStableDiffusionLocal-main\models\v1-5-pruned-emaonly.ckpt
..loading model
LatentDiffusion: Running in eps-prediction mode
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 768 and using 8 heads.
DiffusionWrapper has 859.52 M params.
making attention of type 'vanilla-xformers' with 512 in_channels
building MemoryEfficientAttnBlock with 512 in_channels...
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
making attention of type 'vanilla-xformers' with 512 in_channels
building MemoryEfficientAttnBlock with 512 in_channels...
Exporting Video Frames (1 every 1) frames to E:\tools\DeforumStableDiffusionLocal-main\output\2023-02\Example_DGSpitzer\inputframes...
Converted 25 frames
Loading 25 input frames from E:\tools\DeforumStableDiffusionLocal-main\output\2023-02\Example_DGSpitzer\inputframes and saving video frames to E:\tools\DeforumStableDiffusionLocal-main\output\2023-02\Example_DGSpitzer
Saving animation frames to E:\tools\DeforumStableDiffusionLocal-main\output\2023-02\Example_DGSpitzer
Rendering animation frame 0 of 25
nan 3521683325
Using video init frame E:\tools\DeforumStableDiffusionLocal-main\output\2023-02\Example_DGSpitzer\inputframes\00001.jpg
Global seed set to 3521683325
d:\Users\zoyi\anaconda3\envs\dsd\lib\site-packages\sklearn\cluster_kmeans.py:870: FutureWarning: The default value of n_init will change from 10 to 'auto' in 1.4. Set the value of n_init explicitly to suppress the warning
warnings.warn(
<PIL.Image.Image image mode=RGB size=266x68 at 0x24D0125A020>
d:\Users\zoyi\anaconda3\envs\dsd\lib\site-packages\sklearn\cluster_kmeans.py:870: FutureWarning: The default value of n_init will change from 10 to 'auto' in 1.4. Set the value of n_init explicitly to suppress the warning
warnings.warn(
Traceback (most recent call last):
File "E:\tools\DeforumStableDiffusionLocal-main\run.py", line 488, in
main()
File "E:\tools\DeforumStableDiffusionLocal-main\run.py", line 406, in main
render_input_video(args, anim_args, animation_prompts, root)
File "E:\tools\DeforumStableDiffusionLocal-main\deforum-stable-diffusion\helpers\render.py", line 485, in render_input_video
render_animation(args, anim_args, animation_prompts, root)
File "E:\tools\DeforumStableDiffusionLocal-main\deforum-stable-diffusion\helpers\render.py", line 430, in render_animation
sample, image = generate(args, root, frame_idx, return_latent=False, return_sample=True)
File "E:\tools\DeforumStableDiffusionLocal-main\deforum-stable-diffusion\helpers\generate.py", line 144, in generate
clip_loss_fn = make_clip_loss_fn(root, args)
File "E:\tools\DeforumStableDiffusionLocal-main\deforum-stable-diffusion\helpers\conditioning.py", line 74, in make_clip_loss_fn
target_embeds, weights = parse_clip_prompts(args.clip_prompt)
File "E:\tools\DeforumStableDiffusionLocal-main\deforum-stable-diffusion\helpers\conditioning.py", line 59, in parse_clip_prompts
for prompt in clip_prompt:
TypeError: 'numpy.float64' object is not iterable