TencentARC / t2i-adapter-depth-zoe-sdxl-1.0

huggingface.co
Total runs: 4.3K
24-hour runs: 0
7-day runs: -152
30-day runs: 38
Model's Last Updated: Septembre 08 2023
image-to-image

Introduction of t2i-adapter-depth-zoe-sdxl-1.0

Model Details of t2i-adapter-depth-zoe-sdxl-1.0

T2I-Adapter-SDXL - Depth-Zoe

T2I Adapter is a network providing additional conditioning to stable diffusion. Each t2i checkpoint takes a different type of conditioning as input and is used with a specific base stable diffusion checkpoint.

This checkpoint provides conditioning on depth for the StableDiffusionXL checkpoint. This was a collaboration between Tencent ARC and Hugging Face .

Model Details
  • Developed by: T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models

  • Model type: Diffusion-based text-to-image generation model

  • Language(s): English

  • License: Apache 2.0

  • Resources for more information: GitHub Repository , Paper .

  • Model complexity:

    SD-V1.4/1.5 SD-XL T2I-Adapter T2I-Adapter-SDXL
    Parameters 860M 2.6B 77 M 77/79 M
  • Cite as:

    @misc{ title={T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models}, author={Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, Ying Shan, Xiaohu Qie}, year={2023}, eprint={2302.08453}, archivePrefix={arXiv}, primaryClass={cs.CV} }

Checkpoints
Model Name Control Image Overview Control Image Example Generated Image Example
TencentARC/t2i-adapter-canny-sdxl-1.0
Trained with canny edge detection
A monochrome image with white edges on a black background.
TencentARC/t2i-adapter-sketch-sdxl-1.0
Trained with PidiNet edge detection
A hand-drawn monochrome image with white outlines on a black background.
TencentARC/t2i-adapter-lineart-sdxl-1.0
Trained with lineart edge detection
A hand-drawn monochrome image with white outlines on a black background.
TencentARC/t2i-adapter-depth-midas-sdxl-1.0
Trained with Midas depth estimation
A grayscale image with black representing deep areas and white representing shallow areas.
TencentARC/t2i-adapter-depth-zoe-sdxl-1.0
Trained with Zoe depth estimation
A grayscale image with black representing deep areas and white representing shallow areas.
TencentARC/t2i-adapter-openpose-sdxl-1.0
Trained with OpenPose bone image
A OpenPose bone image.
Example

To get started, first install the required dependencies:

pip install -U git+https://github.com/huggingface/diffusers.git
pip install -U controlnet_aux==0.0.7 timm==0.6.12 # for conditioning models and detectors
pip install transformers accelerate safetensors
  1. Images are first downloaded into the appropriate control image format.
  2. The control image and prompt are passed to the StableDiffusionXLAdapterPipeline .

Let's have a look at a simple example using the Depth-zoe Adapter .

  • Dependency
from diffusers import StableDiffusionXLAdapterPipeline, T2IAdapter, EulerAncestralDiscreteScheduler, AutoencoderKL
from diffusers.utils import load_image, make_image_grid
from controlnet_aux import ZoeDetector
import torch

# load adapter
adapter = T2IAdapter.from_pretrained(
  "TencentARC/t2i-adapter-depth-zoe-sdxl-1.0", torch_dtype=torch.float16, varient="fp16"
).to("cuda")

# load euler_a scheduler
model_id = 'stabilityai/stable-diffusion-xl-base-1.0'
euler_a = EulerAncestralDiscreteScheduler.from_pretrained(model_id, subfolder="scheduler")
vae=AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", torch_dtype=torch.float16)
pipe = StableDiffusionXLAdapterPipeline.from_pretrained(
    model_id, vae=vae, adapter=adapter, scheduler=euler_a, torch_dtype=torch.float16, variant="fp16", 
).to("cuda")
pipe.enable_xformers_memory_efficient_attention()

zoe_depth = ZoeDetector.from_pretrained(
    "valhalla/t2iadapter-aux-models", filename="zoed_nk.pth", model_type="zoedepth_nk"
).to("cuda")
  • Condition Image
url = "https://huggingface.co/Adapter/t2iadapter/resolve/main/figs_SDXLV1.0/org_zeo.jpg"
image = load_image(url)
image = zoe_depth(image, gamma_corrected=True, detect_resolution=512, image_resolution=1024)

  • Generation
prompt = "A photo of a orchid, 4k photo, highly detailed"
negative_prompt = "anime, cartoon, graphic, text, painting, crayon, graphite, abstract, glitch, deformed, mutated, ugly, disfigured"

gen_images = pipe(
  prompt=prompt,
  negative_prompt=negative_prompt,
  image=image,
  num_inference_steps=30,
  adapter_conditioning_scale=1,
  guidance_scale=7.5,  
).images[0]
gen_images.save('out_zoe.png')

Training

Our training script was built on top of the official training script that we provide here .

The model is trained on 3M high-resolution image-text pairs from LAION-Aesthetics V2 with

  • Training steps: 25000
  • Batch size: Data parallel with a single gpu batch size of 16 for a total batch size of 256 .
  • Learning rate: Constant learning rate of 1e-5 .
  • Mixed precision: fp16

Runs of TencentARC t2i-adapter-depth-zoe-sdxl-1.0 on huggingface.co

4.3K
Total runs
0
24-hour runs
-152
3-day runs
-152
7-day runs
38
30-day runs

More Information About t2i-adapter-depth-zoe-sdxl-1.0 huggingface.co Model

More t2i-adapter-depth-zoe-sdxl-1.0 license Visit here:

https://choosealicense.com/licenses/apache-2.0

t2i-adapter-depth-zoe-sdxl-1.0 huggingface.co

t2i-adapter-depth-zoe-sdxl-1.0 huggingface.co is an AI model on huggingface.co that provides t2i-adapter-depth-zoe-sdxl-1.0's model effect (), which can be used instantly with this TencentARC t2i-adapter-depth-zoe-sdxl-1.0 model. huggingface.co supports a free trial of the t2i-adapter-depth-zoe-sdxl-1.0 model, and also provides paid use of the t2i-adapter-depth-zoe-sdxl-1.0. Support call t2i-adapter-depth-zoe-sdxl-1.0 model through api, including Node.js, Python, http.

t2i-adapter-depth-zoe-sdxl-1.0 huggingface.co Url

https://huggingface.co/TencentARC/t2i-adapter-depth-zoe-sdxl-1.0

TencentARC t2i-adapter-depth-zoe-sdxl-1.0 online free

t2i-adapter-depth-zoe-sdxl-1.0 huggingface.co is an online trial and call api platform, which integrates t2i-adapter-depth-zoe-sdxl-1.0's modeling effects, including api services, and provides a free online trial of t2i-adapter-depth-zoe-sdxl-1.0, you can try t2i-adapter-depth-zoe-sdxl-1.0 online for free by clicking the link below.

TencentARC t2i-adapter-depth-zoe-sdxl-1.0 online free url in huggingface.co:

https://huggingface.co/TencentARC/t2i-adapter-depth-zoe-sdxl-1.0

t2i-adapter-depth-zoe-sdxl-1.0 install

t2i-adapter-depth-zoe-sdxl-1.0 is an open source model from GitHub that offers a free installation service, and any user can find t2i-adapter-depth-zoe-sdxl-1.0 on GitHub to install. At the same time, huggingface.co provides the effect of t2i-adapter-depth-zoe-sdxl-1.0 install, users can directly use t2i-adapter-depth-zoe-sdxl-1.0 installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

t2i-adapter-depth-zoe-sdxl-1.0 install url in huggingface.co:

https://huggingface.co/TencentARC/t2i-adapter-depth-zoe-sdxl-1.0

Url of t2i-adapter-depth-zoe-sdxl-1.0

t2i-adapter-depth-zoe-sdxl-1.0 huggingface.co Url

Provider of t2i-adapter-depth-zoe-sdxl-1.0 huggingface.co

TencentARC
ORGANIZATIONS

Other API from TencentARC

huggingface.co

Create photos, paintings and avatars for anyone in any style within seconds.

Total runs: 35.0K
Run Growth: -43.4K
Growth Rate: -124.12%
Updated: Juillet 22 2024
huggingface.co

Total runs: 140
Run Growth: -78
Growth Rate: -55.71%
Updated: Décembre 16 2024
huggingface.co

Total runs: 114
Run Growth: 22
Growth Rate: 19.30%
Updated: Novembre 29 2024
huggingface.co

Total runs: 19
Run Growth: 11
Growth Rate: 57.89%
Updated: Décembre 10 2024
huggingface.co

Total runs: 5
Run Growth: -1
Growth Rate: -20.00%
Updated: Décembre 30 2024
huggingface.co

Total runs: 5
Run Growth: -2
Growth Rate: -40.00%
Updated: Décembre 30 2024
huggingface.co

Total runs: 4
Run Growth: -6
Growth Rate: -150.00%
Updated: Décembre 30 2024
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: Juin 29 2024
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: Août 20 2023
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: Décembre 16 2024
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: Août 28 2023
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: Décembre 20 2024
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: Août 13 2024
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: Décembre 17 2024
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: Avril 11 2024
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: Octobre 08 2022
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: Janvier 20 2024