Transform Your Videos: A Step-by-Step Guide to Using Comfy UI for Stunning AI Animations in 2025
Introduction
The quality and consistency of AI animations have improved significantly in the past two years. In this blog, I will show you the easiest way to get your tools ready and share with you the settings to transform your videos into anything you can imagine. These AI animation methods are only going to get better, so make sure you read till the end to stay and learn how to use them.
To get started, you will need to install ComfyUI. Click here to Download ComfyUI
There, you need to scroll down and find the direct link to download. Right-click on it and choose "Save link as." You can save it anywhere on your computer; I'm going to choose desktop.
Then extract the archive, open the extracted folder, and open ComfyUI. Inside, navigate to custom nodes folder, and here you will also need to install the ComfyUI manager. To do that, select the folder path, type CMD, and hit enter to open the command prompt window.
Then paste this command with the link to the ComfyUI manager. (git clone https://github.com/ltdrdata/ComfyUI-Manager).
Once it's done, you will find that the ComfyUI manager has been installed under custom nodes folder. Go back to the main folder and run this file 'run_nvidia_gpu' for Nvidia users running on Nvidia GPUs or 'run_CPU' which will launch Comfy UI on your browser.
Now, if you already had ComfyUI installed before reading this blog post, open the manager and click on "Update all" to make sure you're running the latest version.
Essential Downloads
To get started with video animation work, head over to this guide on CivitAI. Click here to access the guide on CivitAI. Open the attachment tab and download the file that says IPAdapter Batch Unfold for hotshot with SDXL.json; this is a JSON file that you can drag and drop onto your Comfy interface to load the base workflow.
You will most likely get an error indicating that this workflow is using some nodes that are not installed on your computer. To fix that, open the ComfyUI manager and click on "Install missing custom nodes." You will have a list of extensions that need to be downloaded; go ahead and install all of them one by one, and once you're done, click on restart to relaunch ComfyUI, and now you should be good to go.
Key Files to Download
We have a few more essential files to download, starting with the main AI model that will define the style of your output. In this video, I'm going with ProtoVision XL. Click here to access ProtoVision XL. Right-click on the download button, choose "Save link as," go to the ComfyUI folder, open models folder, check for checkpoints folder, and save the file there.
The second file you need to download is the SDXL- VAE module. Click here to access the SDXL-VAE module. Check for sdxl_vae.safetensors, Right-click on the little download icon, choose "Save link as," and this time go to the VAE folder inside the models folder and save the sdxl_vae.safetensors model there.
You will also need to download this image encoder. Click here to access the Image Encoder .Simply look for this file 'model.safetensors', right-click here, choose "Save link as," go to ComfyUI folder, open the models folder, open Clip_Visions folder, and because the name is too generic, I'm going to change it to Image Encoder but you can leave yours as model, if you wish. Then hit save.
Next, you will need to download this Control Net model - t2i-adapter-depth-midas-sdxl-1.0/; Click here to access the Control Net Model. Here you have two versions of the same model to choose from, which are diffusion_pytorch_model.fp16.safetensors and diffusion_pytorch_model.safensors. Go ahead and download both and save them under models control net. Go to ComfyUI folder, select models folder and select controlnet folder. Save both of them in the controlnet folder.
Last but not least, you will need to download this Hot Shot Motion model; Click here is access the Hot Shot Motion Model. This too has two different versions to choose from, which are hsxl_temporal_layers.f16.safertensors and hsxl_temporal_layers.safetensors, so make sure you download both and place them under ComfyUI folder, open custom_nodes folder, select and open ComfyUI-AnimateDiff-Evolved folder, open models folder and save them there.
Configuring Comfy UI
Launching ComfyUI Interface
Now let's go back to the ComfyUI interface and start working on the settings. First thing you need to do here is load the video file that you want to transform. If you don't have any clips, you can visit any website that offers free stock videos without watermark and copyright free, use a 9:16 ratio, or record yourself dancing with your camera.
I have a few clips that I shortlisted, and I'm going to try and transform this video right here. To do so, hold shift and right-click on the video you want to use, click on 'copy as path', and paste it over in this input. Make sure you delete the quotes, then hit okay. Here you can increase the select every nth frame setting to tell the AI how many frames to process. For example, if you set this to two, the AI will only process every other frame; this is useful if you want to cut down the processing time. You can later use another AI tool such as Video AI to interpolate the video and make it smoother. I'm going to stick to one here because I want to process every single frame.
Right below, you can choose to upscale the processed animation to a higher resolution, which will drastically improve the quality and is supposed to run a little faster than running the animation at a high resolution from the beginning. So I'm going to upscale mine to 1080p. Below that, use this node to select the AI model that you want to use to stylize. If you've downloaded the model - protovisionXLHighFidelity3D_releaseV660Bakedvae.safetensors but still can't see it on the drop-down menu, make sure you click on refresh and it should appear on the list. I'm going to select the Protovision XL model. Right below, you can load the SDXL VAE model.
Now let's move down here to load the IP adapter model and the image encoder right below it. Right next to that, we have another IP adapter node which contains some of the most important settings in this process, and the correct values will highly depend on your input video. I've managed to get really good results by only changing the weight and the noise; you can start with the weight set to 0.2 and the noise set to 0.3.
Changing these two will have a significant effect on your output, so I highly recommend that you play around with the values.
Next, let's move over to the control net nodes; here make sure you load the control net model. To my understanding, choosing the FP16 version - diffusion_pytorch_model.fp16.safetensors will give you less precision but it will result in running a little faster. Since I want the best quality possible, I'm going to go with the normal version - diffusion_pytorch_model.safetensors.
The control net strength will define how closely the animation should follow the original structure of your video, and I usually set this to one.
.png)
Down here in the Anime Diff node, make sure you load the Hot Shot Motion model.
Now move on to another very important node in this process, and that is the K sampler. Let's change the seed control, control_after_generate to randomize; increasing the steps will usually result in better quality outputs—let's set it to 30. The CFG value will determine how closely the output should follow your prompt; the lower it is, the more creative it will be, and a value of 8 is usually a good start for the sample. I like to use dpmpp_3m_ sde_gpu; let's change the scheduler to karras.
I found that the start at step setting has a significant impact on the transformation level; I usually set it somewhere between 2 and 15 but 6 is a safe value to start with — and the higher this is, the less transformation you will have, but of course this works in combination with all the other settings that we talked about so keep that in mind.
Down below we have what is probably the most important input in this whole workflow, and that is prompting. You have two boxes; the green one is meant for you to input positive prompts — you can use this to describe the final output that you want to see. I tend to describe the subject and I like to be precise with clothing and environment to help achieve better consistency.
The two input boxes are supposed to serve different purposes but I found that using the same prompt in both boxes is a safe option to go with.
(Positive Prompt)
Moving on to the Video Combine node; here you can set up your export settings. I usually match the frame rate to that of my original video which in this case is 25.
On the far right of our workflow, we have upscaling nodes here—you don't have to change much; you can keep upscaling K sampler settings at default—and at the end of that there's another Video Combine node; again, you can match it with same frame rate as your original video. I will leave my frane_rate to 25. You can also customize how your output videos are named or change their final video format in filename_prefix textbox.
Once you get all your inputs and settings done, you can go ahead and click on Queue Prompt to start processing. You will see that ComfyUI is going through nodes one by one; The K Sampler node will be taking the heaviest load of whole process.
Once that's done, you will have a preview in Video Combine node; this is 720p version prior upscaling and it already looks pretty good. ComfyUI will move on upscaling video right after that and you'll still be able see progress here as well. Once that's done, final Video Combine node will display preview of your upscaled output.
I'm honestly very happy with the results here—it's everything I imagined the video would look like. Chances are, if you're running this for the first time, you won't get the desired output right away. This is where experimenting comes in. I highly recommend going back, tweaking the settings, and executing multiple times until you get a really good output.
To access the generated animations, go to the ComfyUI folder and look for the output folder. You'll find the final upscaled videos as well as individual frames and pre-upscaled outputs stored in different folders. The great thing about this setup is that you can take any output, drag it over, and drop it onto the ComfyUI interface to load the exact same settings used in that video.
