Finally, better depth for Stable Diffusion ControlNet!

Is this the best way to control your AI images?

Professional users of AI image generation will know and love Stable Diffusion with ControlNet—if you don’t, scroll down a bit! But they’ll also be disappointed by the details that can get lost.

No longer! Combine these two tricks and get perfect SDXL results: DepthAnything V2 and the new Xinsir Depth Control Model. Even hair isn’t an issue!

What to do first

If you’ve yet to discover the world of open source image generation with Stable Diffusion, I recommend you learn the fundamentals in my tutorial.

If you already know the basics of Stable Diffusion, skip ahead to Minute 10:32 to learn about ControlNet!

Once you’ve done that, you’re ready to install these new ControlNet models and step up your game.

DepthAnything V2

Every successful ControlNet workflow consists of two things: the preprocessor, and the actual ControlNet model. The preprocessor recognizes depth in your images. This used to be pretty imprecise. You couldn’t possibly have preserved details like hair, without going to great lengths and processing power.

That’s changed with the new DepthAnything V2. It’s available as an extension for the Auto1111 and Forge WebUIs for now, to be found in the “Extras” tab once you’ve installed it. Refer to my above tutorial for instructions on installing extensions like this.

Run your image through DepthAnything V2, then drag and drop the black and white result into ControlNet. Make sure to set the preprocessor to “none”, as you’ve already completed that step.

Xinsir – A Better ControlNet Depth Model

On June 26th, new ControlNet models dropped. To use them, download the linked file named “diffusion_pytorch_model.safetensors”, rename it to something like “SDXL_control_depth_xinsir” and place it in your models/controlnet folder.

Once you’ve done that, you can refresh your UI and find the model where you select your ControlNets! If you have any trouble finding this, refer to the tutorial above. You’ll get it running in no time!

Do you want to learn more from the cutting edge of Image Gen AI?