ai-forever / Kandinsky3.0

huggingface.co
Total runs: 0
24-hour runs: 0
7-day runs: 0
30-day runs: 0
Model's Last Updated: Novembro 22 2023

Introduction of Kandinsky3.0

Model Details of Kandinsky3.0

Kandinsky-3: Text-to-image diffusion model

Post | Generate | Telegram-bot | [Report]

Description:

Kandinsky 3.0 is an open-source text-to-image diffusion model built upon the Kandinsky2-x model family. In comparison to its predecessors, Kandinsky 3.0 incorporates more data and specifically related to Russian culture, which allows to generate pictures related to Russin culture. Furthermore, enhancements have been made to the text understanding and visual quality of the model, achieved by increasing the size of the text encoder and Diffusion U-Net models, respectively.

For more information: details of training, example of generations check out our post . The english version will be released in a couple of days.

Architecture details:

Architecture consists of three parts:

  • Text encoder Flan-UL2 (encoder part) - 8.6B
  • Latent Diffusion U-Net - 3B
  • MoVQ encoder/decoder - 267M
Models

We release our two models:

  • Base: Base text-to-image diffusion model. This model was trained over 2M steps on 400 A100
  • Inpainting: Inpainting version of the model. The model was initialized from final checkpoint of base model and trained 250k steps on 300 A100.

Weights of the model are loaded internally but if want to change them one can use the following example:

from huggingface_hub import hf_hub_download
from kandinsky3 import get_T2I_unet, get_T5encoder, get_movq, Kandinsky3T2IPipeline

unet_path = hf_hub_download(
        repo_id="ai-forever/Kandinsky3.0", filename='weights/kandinsky3.pt')
)

movq_path = hf_hub_download(
          repo_id="ai-forever/Kandinsky3.0", filename='weights/movq.pt')
)
unet, null_embedding, projections_state_dict = get_T2I_unet(device, unet_path, fp16=fp16)
processor, condition_encoders = get_T5encoder(device, text_encode_path, projections_state_dict, fp16=fp16)
movq = get_movq(device, movq_path, fp16=fp16)
t2i_pipe =  Kandinsky3T2IPipeline(device, unet, null_embedding, processor, condition_encoders, movq, fp16=fp16)
from huggingface_hub import hf_hub_download
from kandinsky3 import get_inpainting_unet, get_T5encoder, get_movq, Kandinsky3InpaintingPipeline

inpainting_unet_path = hf_hub_download(
          repo_id="ai-forever/Kandinsky3.0", filename='weights/kandinsky3_inpainting.pt', cache_dir=cache_dir
)
movq_path = hf_hub_download(
          repo_id="ai-forever/Kandinsky3.0", filename='weights/movq.pt')
)

unet, null_embedding, projections_state_dict = get_inpainting_unet(device, unet_path, fp16=fp16)
processor, condition_encoders = get_T5encoder(device, text_encode_path, projections_state_dict, fp16=fp16)
movq = get_movq(device, movq_path, fp16=False) #MoVQ ooesn't work properly in fp16 on inpainting
pipe = Kandinsky3InpaintingPipeline(device, unet, null_embedding, processor, condition_encoders, movq, fp16=fp16)
Installing

To install repo first one need to create conda environment:

conda create -n kandinsky -y python=3.8;
source activate kandinsky;
pip install torch==1.10.1+cu111 torchvision==0.11.2+cu111 torchaudio==0.10.1 -f https://download.pytorch.org/whl/cu113/torch_stable.html;
pip install -r requirements.txt;

The exact dependencies is got using pip freeze and can be found in exact_requirements.txt

How to use:

Check our jupyter notebooks with examples in ./examples folder

1. text2image
from kandinsky3 import get_T2I_pipeline

t2i_pipe = get_T2I_pipeline('cuda', fp16=True)

image = t2i_pipe( "A cute corgi lives in a house made out of sushi.")
2. inpainting
from kandinsky3 import get_inpainting_pipeline

inp_pipe = get_inpainting_pipeline('cuda', fp16=True)

image = ... # PIL Image
mask = ... # Numpy array (HxW). Set 1 where image should be masked
image = inp_pipe( "A cute corgi lives in a house made out of sushi.", image, mask)
Examples of generations

"A beautiful landscape outdoors scene in the crochet knitting art style, drawing in style by Alfons Mucha" "gorgeous phoenix, cosmic, darkness, epic, cinematic, moonlight, stars, high - definition, texture,Oscar-Claude Monet" "a yellow house at the edge of the danish fjord, in the style of eiko ojala, ingrid baars, ad posters, mountainous vistas, george ault, realistic details, dark white and dark gray, 4k" "dragon fruit head, upper body, realistic, illustration by Joshua Hoffine Norman Rockwell, scary, creepy, biohacking, futurism, Zaha Hadid style"
"Amazing playful nice cute strawberry character, dynamic poze, surreal fantazy garden background, gorgeous masterpice, award winning photo, soft natural lighting, 3d, Blender, Octane render, tilt - shift, deep field, colorful, I can't believe how beautiful this is, colorful, cute and sweet baby - loved photo" "beautiful fairy-tale desert, in the sky a wave of sand merges with the milky way, stars, cosmism, digital art, 8k" "Car, mustang, movie, person, poster, car cover, person, in the style of alessandro gottardo, gold and cyan, gerald harvey jones, reflections, highly detailed illustrations, industrial urban scenes"" "cloud in blue sky, a red lip, collage art, shuji terayama, dreamy objects, surreal, criterion collection, showa era, intricate details, mirror"

Authors

Runs of ai-forever Kandinsky3.0 on huggingface.co

0
Total runs
0
24-hour runs
0
3-day runs
0
7-day runs
0
30-day runs

More Information About Kandinsky3.0 huggingface.co Model

More Kandinsky3.0 license Visit here:

https://choosealicense.com/licenses/apache-2.0

Kandinsky3.0 huggingface.co

Kandinsky3.0 huggingface.co is an AI model on huggingface.co that provides Kandinsky3.0's model effect (), which can be used instantly with this ai-forever Kandinsky3.0 model. huggingface.co supports a free trial of the Kandinsky3.0 model, and also provides paid use of the Kandinsky3.0. Support call Kandinsky3.0 model through api, including Node.js, Python, http.

ai-forever Kandinsky3.0 online free

Kandinsky3.0 huggingface.co is an online trial and call api platform, which integrates Kandinsky3.0's modeling effects, including api services, and provides a free online trial of Kandinsky3.0, you can try Kandinsky3.0 online for free by clicking the link below.

ai-forever Kandinsky3.0 online free url in huggingface.co:

https://huggingface.co/ai-forever/Kandinsky3.0

Kandinsky3.0 install

Kandinsky3.0 is an open source model from GitHub that offers a free installation service, and any user can find Kandinsky3.0 on GitHub to install. At the same time, huggingface.co provides the effect of Kandinsky3.0 install, users can directly use Kandinsky3.0 installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

Kandinsky3.0 install url in huggingface.co:

https://huggingface.co/ai-forever/Kandinsky3.0

Url of Kandinsky3.0

Kandinsky3.0 huggingface.co Url

Provider of Kandinsky3.0 huggingface.co

ai-forever
ORGANIZATIONS

Other API from ai-forever

huggingface.co

Total runs: 525.5K
Run Growth: 507.4K
Growth Rate: 96.56%
Updated: Novembro 03 2023
huggingface.co

Total runs: 10.6K
Run Growth: 1.5K
Growth Rate: 13.75%
Updated: Dezembro 05 2023
huggingface.co

Total runs: 8.2K
Run Growth: 5.1K
Growth Rate: 59.45%
Updated: Dezembro 29 2024
huggingface.co

Total runs: 5.9K
Run Growth: 3.5K
Growth Rate: 60.26%
Updated: Dezembro 11 2023
huggingface.co

Total runs: 2.3K
Run Growth: -408
Growth Rate: -17.86%
Updated: Dezembro 05 2023
huggingface.co

Total runs: 1.3K
Run Growth: -6.3K
Growth Rate: -493.39%
Updated: Dezembro 05 2023
huggingface.co

Total runs: 315
Run Growth: 165
Growth Rate: 52.05%
Updated: Janeiro 26 2023
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: Junho 08 2023
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: Dezembro 24 2021
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: Setembro 21 2021