CogVideoX Advanced WorkFlow: Lora | ControlNet | Text to Video | Image to Video

I’ve been testing CogVideoX lately—the tool that turns still images into short video clips—and the latest update adds some solid improvements. It’s especially handy for Linux users, but there’s plenty here for everyone. Here’s what stood out.

One of the bigger headaches with AI video generation is when clips end too abruptly. FreeNoise tackles this by extending the video with new, coherent scenes. It’s not just tacking on extra frames; it actually continues the motion in a way that (mostly) makes sense.

For example, I fed it a 5-second clip of a cat jumping onto a table. Instead of cutting off mid-air, FreeNoise added a few more seconds of the cat landing and swatting at something off-camera. It’s not perfect—sometimes the added scenes get a little surreal—but it’s way better than a hard stop.

If you’re running CogVideoX on Linux, SageAttention makes a noticeable difference. Render times dropped by about 20-30% in my tests, which is a big deal when you’re tweaking prompts and regenerating clips. It’s not magic—heavier workloads still take time—but it’s a solid improvement.

ControlNet isn’t new, but the latest version gives you two distinct ways to guide the generation:

• Canny: Best for sharp, defined edges. Works well if you want a stylized or high-contrast look.
• HED: Captures softer shapes and details, which helps with more organic subjects like faces or landscapes.

I tried both on the same input image, and the difference is clear—Canny keeps things crisp, while HED preserves subtler textures. Neither’s perfect, but having the choice helps.

The cogvideox-5b-fun-lora-bladerunnerv1 model is now available, and it’s… fun. I loaded it up in ComfyUI and got some surprisingly dynamic results, especially with cyberpunk-style prompts. It’s not going to replace the base model for serious work, but for experimenting, it’s worth a look.

• ControlNet models: GitHub repo
• LoRA model: Hugging Face