DeepSeek AI Janus Pro in Comfy UI for Image and Video Generation Workflow

Are you looking to explore the capabilities of AI for image and video generation? DeepSeek AI’s Janus Pro is a powerful multimodal model that can understand and generate images, making it a great tool for creative workflows. In this guide, I’ll walk you through how to set up and use Janus Pro in Comfy UI, step by step. Whether you’re a beginner or an experienced user, this tutorial will help you get started with AI-driven image and video creation.

Janis Pro on Hugging Face

What is DeepSeek Janis Pro?

DeepSeek Janus Pro is a multimodal AI model that combines vision and language capabilities. It can:

Generate images from text prompts (text-to-image).
Analyze and describe images (image-to-text).
Create video prompts for AI video generation.

This makes it a versatile tool for artists, designers, and content creators who want to automate and enhance their creative processes.

Step 1: Installing Janus Pro in Comfy UI

To get started, you’ll need to install the Janus Pro custom node in Comfy UI. Here’s how:

Open Comfy UI and search for “Janus Pro” in the custom nodes section.
Click the install button and select the latest version of the node.
After installation, restart Comfy UI. Instead of using the restart button, stop the server and open a command prompt window.
Navigate to the custom nodes directory and locate the Janus Pro folder.
Run the pip install command to install the required dependencies. For example:

pip install -r requirements.txt

Once the installation is complete, restart Comfy UI and verify that the Janis Pro node is successfully imported.

Step 2: Downloading and Setting Up the Models

Next, you’ll need to download the Janus Pro model files. Here’s how to do it:

Visit the official DeepSeek AI model card page on Hugging Face.
Download the model files for the version you want to use (e.g., Janis Pro 7B).
Create a folder in your Comfy UI directory with the following path:

ComfyUI/models/Janis Pro/Janis Pro 7B

Place all the downloaded model files in this folder.

Now, your Comfy UI is ready to use Janis Pro for image and video generation.

Step 3: Using Janus Pro for Image Generation

Janus Pro’s image generation feature allows you to create images from text prompts. Here’s how to set it up:

In Comfy UI, right-click and select the Janus Pro nodes: Model Loader, Image Generation, and Image Understanding.
Connect the Model Loader to the Image Generation node.
Enter a text prompt, such as “a beautiful photo of a flower,” in the Image Generation node.
Adjust settings like seed number, batch size, CFG scale, temperature, and top P to control the output.
Drag the Preview Image node to see the generated image.

For example, I used the prompt “a man sitting in a red T-shirt with a laptop” and got a realistic image with a dark background. The results may not be as detailed as specialized diffusion models, but they’re great for quick drafts or concept art.

Step 4: Using Janus Pro for Image Understanding

Janis Pro’s image understanding feature is perfect for analyzing and describing images. Here’s how to use it:

Connect the Model Loader to the Image Understanding node.
Load an image into the Image Understanding node.
Enter a specific question, like “What kind of person is in this image?”
The AI will generate a detailed description of the image, focusing on the elements you asked about.

For instance, I used an image of a man in a red T-shirt, and Janis Pro described him as “a man sitting in a red T-shirt with a laptop in front of him.” This description can then be used as a text prompt for further image or video generation.

Step 5: Creating Video Prompts with Janus Pro

Janus Pro can also generate text prompts for AI video generation. Here’s how:

Use the Image Understanding node to analyze a reference image.
Ask the AI to create video text prompts, including actions or camera effects. For example:

Write AI video text prompts using the style of this image as a reference and include some actions or camera effects.

The AI will generate detailed video prompts, such as “a whale swimming forward in space.”
Use these prompts in video generation tools like LTX or AnimateDiff to create videos.

I tested this with an image of a whale and got a video of a whale swimming through space. The results were impressive, with smooth motions and accurate style transfer.

Step 6: Video-to-Video Workflow with Janus Pro

For video-to-video workflows, Janus Pro can replace models like Florence 2. Here’s how:

Load a source video and a reference image into Comfy UI.
Use Janis Pro to generate text prompts based on the reference image.
Connect the text prompts to the video generation nodes.
Adjust settings like strength and style to match the reference image.

For example, I used a reference image of an ancient Chinese scene with a lady in a red dress and generated a video with a similar style. The output video had cohesive motions and colorations, making it a great tool for style transfer.

Why Choose Janus Pro?

Janus Pro stands out because of its ability to:

Combine image understanding and generation in one model.
Generate detailed text prompts for videos.
Work seamlessly with Comfy UI for creative workflows.

While it may not replace specialized diffusion models for high-resolution images, it’s a great tool for quick drafts, concept art, and video prompts.

Conclusion

DeepSeek AI’s Janus Pro is a versatile and powerful tool for image and video generation. By integrating it with Comfy UI, you can streamline your creative workflows and automate tasks like text prompt generation and style transfer. Whether you’re an artist, designer, or content creator, Janus Pro offers practical solutions for your projects.

I hope this guide helps you get started with Janus Pro in Comfy UI. If you have any questions or need further assistance, feel free to reach out. Happy creating!

Download Workflows

🤖 Hey, welcome! Thanks for visiting comfyuiblog.com