ai-forever / KandinskyVideo

huggingface.co
Total runs: 0
24-hour runs: 0
7-day runs: 0
30-day runs: 0
Model's Last Updated: November 22 2023

Introduction of KandinskyVideo

Model Details of KandinskyVideo

Kandinsky Video — a new text-to-video generation model

SoTA quality among open-source solutions

This repository is the official implementation of Kandinsky Video model

Paper | Project | Hugging Face Spaces | Telegram-bot | Habr post


Kandinsky Video is a text-to-video generation model, which is based on the FusionFrames architecture, consisting of two main stages: keyframe generation and interpolation. Our approach for temporal conditioning allows us to generate videos with high-quality appearance, smoothness and dynamics.

Pipeline


The encoded text prompt enters the U-Net keyframe generation model with temporal layers or blocks, and then the sampled latent keyframes are sent to the latent interpolation model in such a way as to predict three interpolation frames between two keyframes. A temporal MoVQ-GAN decoder is used to get the final video result.

Architecture details

  • Text encoder (Flan-UL2) - 8.6B
  • Latent Diffusion U-Net3D - 4.0B
  • MoVQ encoder/decoder - 256M
How to use

Check our jupyter notebooks with examples in ./examples folder

1. text2video
from video_kandinsky3 import get_T2V_pipeline

t2v_pipe = get_T2V_pipeline('cuda', fp16=True)

pfps = 'medium' # ['low', 'medium', 'high']
video = t2v_pipe(
    'a red car is drifting on the mountain road, close view, fast movement',
    width=640, height=384, fps=fps
)
Results
"A car moving on the road from the sea to the mountains" "A red car drifting, 4k video" "Chemistry laboratory, chemical explosion, 4k" "Erupting volcano raw power, molten lava, and the forces of the Earth"
"Luminescent jellyfish swims underwater, neon, 4k" "Majestic waterfalls in a lush rainforest power, mist, and biodiversity" "White ghost flies through a night clearing, 4k" "Wildlife migration herds on the move, crossing landscapes in harmony"
"Majestic humpback whale breaching power, grace, and ocean spectacle" "Evoke the sense of wonder in a time-lapse journey through changing seasons" "Explore the fascinating world of underwater creatures in a visually stunning sequence" "Polar ice caps the pristine wilderness of the Arctic and Antarctic"
"Rolling waves on a sandy beach relaxation, rhythm, and coastal beauty" "Sloth in slow motion deliberate movements, relaxation, and arboreal life" "Time-lapse of a flower blooming growth, beauty, and the passage of time" "Craft a heartwarming narrative showcasing the bond between a human and their loyal pet companion"

Authors

BibTeX

If you use our work in your research, please cite our publication:

TBD

Runs of ai-forever KandinskyVideo on huggingface.co

0
Total runs
0
24-hour runs
0
3-day runs
0
7-day runs
0
30-day runs

More Information About KandinskyVideo huggingface.co Model

More KandinskyVideo license Visit here:

https://choosealicense.com/licenses/apache-2.0

KandinskyVideo huggingface.co

KandinskyVideo huggingface.co is an AI model on huggingface.co that provides KandinskyVideo's model effect (), which can be used instantly with this ai-forever KandinskyVideo model. huggingface.co supports a free trial of the KandinskyVideo model, and also provides paid use of the KandinskyVideo. Support call KandinskyVideo model through api, including Node.js, Python, http.

ai-forever KandinskyVideo online free

KandinskyVideo huggingface.co is an online trial and call api platform, which integrates KandinskyVideo's modeling effects, including api services, and provides a free online trial of KandinskyVideo, you can try KandinskyVideo online for free by clicking the link below.

ai-forever KandinskyVideo online free url in huggingface.co:

https://huggingface.co/ai-forever/KandinskyVideo

KandinskyVideo install

KandinskyVideo is an open source model from GitHub that offers a free installation service, and any user can find KandinskyVideo on GitHub to install. At the same time, huggingface.co provides the effect of KandinskyVideo install, users can directly use KandinskyVideo installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

KandinskyVideo install url in huggingface.co:

https://huggingface.co/ai-forever/KandinskyVideo

Url of KandinskyVideo

KandinskyVideo huggingface.co Url

Provider of KandinskyVideo huggingface.co

ai-forever
ORGANIZATIONS

Other API from ai-forever

huggingface.co

Total runs: 525.5K
Run Growth: 507.4K
Growth Rate: 96.56%
Updated: November 03 2023
huggingface.co

Total runs: 10.6K
Run Growth: 1.5K
Growth Rate: 13.75%
Updated: December 05 2023
huggingface.co

Total runs: 8.2K
Run Growth: 5.1K
Growth Rate: 59.45%
Updated: December 29 2024
huggingface.co

Total runs: 5.9K
Run Growth: 3.5K
Growth Rate: 60.26%
Updated: December 11 2023
huggingface.co

Total runs: 2.3K
Run Growth: -408
Growth Rate: -17.86%
Updated: December 05 2023
huggingface.co

Total runs: 1.3K
Run Growth: -6.3K
Growth Rate: -493.39%
Updated: December 05 2023
huggingface.co

Total runs: 315
Run Growth: 158
Growth Rate: 50.16%
Updated: January 26 2023
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: December 24 2021
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: June 08 2023
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: September 21 2021