Free for non-commercial use: Individuals and organizations can use the model free of charge for non-commercial use, including scientific research.
Free for commercial use (up to $1M in annual revenue): Startups, small to medium-sized businesses, and creators can use the model for commercial purposes at no cost, as long as their total annual revenue is less than $1M.
Ownership of outputs: Retain ownership of the media generated without restrictive licensing implications.
For organizations with annual revenue more than $1M, please contact us
here
to inquire about an Enterprise License.
Make sure you upgrade to the latest version of diffusers:
pip install -U diffusers
. And then you can run:
import torch
from diffusers import StableDiffusion3ControlNetPipeline, SD3ControlNetModel
from diffusers.utils import load_image
from diffusers.image_processor import VaeImageProcessor
classSD3CannyImageProcessor(VaeImageProcessor):
def__init__(self):
super().__init__(do_normalize=False)
defpreprocess(self, image, **kwargs):
image = super().preprocess(image, **kwargs)
image = image * 255 * 0.5 + 0.5return image
defpostprocess(self, image, do_denormalize=True, **kwargs):
do_denormalize = [True] * image.shape[0]
image = super().postprocess(image, **kwargs, do_denormalize=do_denormalize)
return image
controlnet = SD3ControlNetModel.from_pretrained("stabilityai/stable-diffusion-3.5-large-controlnet-canny", torch_dtype=torch.float16)
pipe = StableDiffusion3ControlNetPipeline.from_pretrained(
"stabilityai/stable-diffusion-3.5-large",
controlnet=controlnet,
torch_dtype=torch.float16
).to("cuda")
pipe.image_processor = SD3CannyImageProcessor()
control_image = load_image("https://huggingface.co/datasets/diffusers/diffusers-images-docs/resolve/main/canny.png")
prompt = "A Night time photo taken by Leica M11, portrait of a Japanese woman in a kimono, looking at the camera, Cherry blossoms"
generator = torch.Generator(device="cpu").manual_seed(0)
image = pipe(
prompt,
control_image=control_image,
controlnet_conditioning_scale=1.0,
guidance_scale=3.5,
num_inference_steps=60,
generator=generator,
max_sequence_length=77,
).images[0]
image.save(f'canny-8b.jpg')
Preprocessing
An input image can be preprocessed for control use following the code snippet below. SD3.5 does not implement this behavior, so we recommend doing so in an external script beforehand.
import torchvision.transforms.functional as F
# assuming img is a PIL image
img = F.to_tensor(img)
img = cv2.cvtColor(img.transpose(1, 2, 0), cv2.COLOR_RGB2GRAY)
img = cv2.Canny(img, 100, 200)
Tips
We recommend starting with a ControlNet strength of 0.8, and adjusting as needed.
Euler sampler and a slightly higher step count (50-60) gives best results.
Pass
--text_encoder_device <device_name>
to load the text encoders directly to VRAM, which can speed up the full inference loop at the cost of extra VRAM usage.
The model was not trained to be factual or true representations of people or events. As such, using the model to generate such content is out-of-scope of the abilities of this model.
Training Data and Strategy
These models were trained on a wide variety of data, including synthetic data and filtered publicly available data.
Safety
We believe in safe, responsible AI practices and take deliberate measures to ensure Integrity starts at the early stages of development. This means we have taken and continue to take reasonable steps to prevent the misuse of Stable Diffusion 3.5 by bad actors. For more information about our approach to Safety please visit our
Safety page
.
Integrity Evaluation
Our integrity evaluation methods include structured evaluations and red-teaming testing for certain harms. Testing was conducted primarily in English and may not cover all possible harms.
Risks identified and mitigations:
Harmful content: We have used filtered data sets when training our models and implemented safeguards that attempt to strike the right balance between usefulness and preventing harm. However, this does not guarantee that all possible harmful content has been removed. All developers and deployers should exercise caution and implement content safety guardrails based on their specific product policies and application use cases.
Misuse: Technical limitations and developer and end-user education can help mitigate against malicious applications of models. All users are required to adhere to our Acceptable Use Policy, including when applying fine-tuning and prompt engineering mechanisms. Please reference the Stability AI Acceptable Use Policy for information on violative uses of our products.
Privacy violations: Developers and deployers are encouraged to adhere to privacy regulations with techniques that respect data privacy.
Acknowledgements
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala, authors of the original
ControlNet paper
.
Lvmin Zhang, who also developed the
Tile ControlNet
, which inspired the Blur ControlNet.
Diffusers
library authors, whose code was referenced during development.
InstantX
team, whose Flux and SD3 ControlNets were also referenced during training.
All early testers and raters of the models, and the Stability AI team.
Contact
Please report any issues with the model or contact us:
stable-diffusion-3.5-large-controlnet-canny huggingface.co is an AI model on huggingface.co that provides stable-diffusion-3.5-large-controlnet-canny's model effect (), which can be used instantly with this stabilityai stable-diffusion-3.5-large-controlnet-canny model. huggingface.co supports a free trial of the stable-diffusion-3.5-large-controlnet-canny model, and also provides paid use of the stable-diffusion-3.5-large-controlnet-canny. Support call stable-diffusion-3.5-large-controlnet-canny model through api, including Node.js, Python, http.
stable-diffusion-3.5-large-controlnet-canny huggingface.co is an online trial and call api platform, which integrates stable-diffusion-3.5-large-controlnet-canny's modeling effects, including api services, and provides a free online trial of stable-diffusion-3.5-large-controlnet-canny, you can try stable-diffusion-3.5-large-controlnet-canny online for free by clicking the link below.
stabilityai stable-diffusion-3.5-large-controlnet-canny online free url in huggingface.co:
stable-diffusion-3.5-large-controlnet-canny is an open source model from GitHub that offers a free installation service, and any user can find stable-diffusion-3.5-large-controlnet-canny on GitHub to install. At the same time, huggingface.co provides the effect of stable-diffusion-3.5-large-controlnet-canny install, users can directly use stable-diffusion-3.5-large-controlnet-canny installed effect in huggingface.co for debugging and trial. It also supports api for free installation.
stable-diffusion-3.5-large-controlnet-canny install url in huggingface.co: