TencentARC / MotionCtrl
Model's Last Updated: 2024年7月19日
Introduction of MotionCtrl
Model Details of MotionCtrl
MotionCtrl Model Card
Project Page | Paper (ArXiv) | Code
🤗 Gradio demo (MotionCtrl+VideoCrafter) | 🤗 Gradio demo (MotionCtrl+SVD)
Introduction
MotionCtrl, when provided with specific camera movements indicated by camera poses or several trajectories, can independently control both the camera and object motion in a generated video. It can be deployed on both Text-to-Video generation models, such as LVDM/VideoCrafter and AnimateDiff , as well as Image-to-Video generation models like SVD .
MotionCtrl + SVD
- Model: motionctrl.pth
- Results:
MotionCtrl + LVDM/VideoCrafter
- Model:
- Results:
MotionCtrl + AnimateDiff
- Model:
- Results:
Usage
- Download directly in this repository OR Download with python script
from huggingface_hub import hf_hub_download
motionctrl_lvdm_ckpt = hf_hub_download(repo_id="TencentARC/MotionCtrl", filename="motionctrl.ckpt", repo_type="model")
motionctrl_svd_ckpt = hf_hub_download(repo_id="TencentARC/MotionCtrl", filename="motionctrl_svd.ckpt", repo_type="model")
- Then generate the controllable video following the instructions in our GitHub repository .
Citation
@inproceedings{wang2023motionctrl,
title={MotionCtrl: A Unified and Flexible Motion Controller for Video Generation},
author={Wang, Zhouxia and Yuan, Ziyang and Wang, Xintao and Chen, Tianshui and Xia, Menghan and Luo, Ping and Shan, Yin},
booktitle={arXiv preprint arXiv:2312.03641},
year={2023}
}
Runs of TencentARC MotionCtrl on huggingface.co
0
Total runs
0
24-hour runs
0
3-day runs
0
7-day runs
0
30-day runs
More Information About MotionCtrl huggingface.co Model
Url of MotionCtrl
MotionCtrl huggingface.co Url
Provider of MotionCtrl huggingface.co
Other API from TencentARC
Create photos, paintings and avatars for anyone in any style within seconds.
Total runs:
35.0K
Run Growth:
-43.4K
Growth Rate:
-124.12%
Total runs:
9.8K
Run Growth:
134
Growth Rate:
1.37%
Total runs:
6.8K
Run Growth:
1.1K
Growth Rate:
16.29%
Total runs:
6.7K
Run Growth:
363
Growth Rate:
5.55%
Total runs:
5.9K
Run Growth:
1.0K
Growth Rate:
17.41%
Total runs:
5.3K
Run Growth:
1.4K
Growth Rate:
27.11%
Total runs:
4.3K
Run Growth:
38
Growth Rate:
0.89%
Total runs:
2.6K
Run Growth:
327
Growth Rate:
12.57%
Total runs:
2.4K
Run Growth:
57
Growth Rate:
2.32%
Total runs:
2.4K
Run Growth:
152
Growth Rate:
6.31%
Total runs:
2.2K
Run Growth:
743
Growth Rate:
34.30%
Total runs:
2.0K
Run Growth:
210
Growth Rate:
10.38%
Total runs:
220
Run Growth:
-181
Growth Rate:
-82.27%
Total runs:
153
Run Growth:
-542
Growth Rate:
-354.25%
Total runs:
57
Run Growth:
35
Growth Rate:
61.40%
Total runs:
51
Run Growth:
-18
Growth Rate:
-35.29%
Total runs:
50
Run Growth:
29
Growth Rate:
60.42%
Total runs:
41
Run Growth:
-103
Growth Rate:
-264.10%
Total runs:
39
Run Growth:
12
Growth Rate:
30.77%
Total runs:
33
Run Growth:
-78
Growth Rate:
-243.75%
Total runs:
27
Run Growth:
9
Growth Rate:
40.91%
Total runs:
24
Run Growth:
3
Growth Rate:
12.50%
Total runs:
19
Run Growth:
3
Growth Rate:
15.79%
Total runs:
18
Run Growth:
9
Growth Rate:
50.00%
Total runs:
12
Run Growth:
2
Growth Rate:
16.67%
Total runs:
12
Run Growth:
2
Growth Rate:
16.67%
Total runs:
12
Run Growth:
2
Growth Rate:
16.67%
Total runs:
9
Run Growth:
2
Growth Rate:
22.22%
Total runs:
7
Run Growth:
-5
Growth Rate:
-71.43%
Total runs:
6
Run Growth:
-2
Growth Rate:
-33.33%
Total runs:
5
Run Growth:
-2
Growth Rate:
-40.00%
Total runs:
5
Run Growth:
-6
Growth Rate:
-120.00%
Total runs:
4
Run Growth:
4
Growth Rate:
100.00%
Total runs:
4
Run Growth:
4
Growth Rate:
100.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%