From the source
From the source
Hugging Face announces the integration of ControlNet into the Diffusers library, enabling users to condition Stable Diffusion generation with spatial contexts like depth maps, segmentation maps, scribbles, and keypoints.
The post introduces the StableDiffusionControlNetPipeline and shows how to use it with various ControlNet models, starting with the Canny model.
From the source
To experiment with ControlNet, Diffusers exposes the StableDiffusionControlNetPipeline similar to the other Diffusers pipelines. Central to the StableDiffusionControlNetPipeline is the controlnet argument which lets us provide a particular trained ControlNetModel instance while keeping the pre-trained diffusion model weights the same.
huggingface.co
