TencentARC / DI-PCG

huggingface.co
Total runs: 0
24-hour runs: 0
7-day runs: 0
30-day runs: 0
Model's Last Updated: December 20 2024

Introduction of DI-PCG

Model Details of DI-PCG

DI-PCG: Diffusion-based Efficient Inverse Procedural Content Generation for High-quality 3D Asset Creation


Wang Zhao 1 , Yan-Pei Cao 2 , Jiale Xu 1 , Yuejiang Dong 1,3 , Ying Shan 1

1 ARC Lab, Tencent PCG 2 VAST 3 Tsinghua University


🚩 Overview

This repository contains code release for our technical report "DI-PCG: Diffusion-based Efficient Inverse Procedural Content Generation for High-quality 3D Asset Creation".

⚙️ Installation

First clone this repository with Infinigen as the submodule:

git clone -r https://github.com/TencentARC/DI-PCG.git
cd DI-PCG
git submodule update --init --recursive

We recommend using anaconda to install the dependencies:

conda create -n di-pcg python=3.10.14
conda activate di-pcg
conda install pytorch==2.4.0 torchvision==0.19.0 torchaudio==2.4.0  pytorch-cuda=11.8 -c pytorch -c nvidia
pip install -r requirements.txt
🚀 Usage

For a quick start, try the huggingface gradio demo here .

Download models

We provide the pretrained diffusion models for chair, vase, table, basket, flower and dandelion. You can download them from model card and put them in ./pretrained_models/ .

Alternatively, the inference script will automatically download the pretrained models for you.

Local gradio demo

To run the gradio demo locally, run:

python app.py
Inference

To run the inference demo, simply use:

python ./scripts/sample_diffusion.py --config ./configs/demo/chair_demo.yaml

This script processes all the chair images in the ./examples/chair folder and saves the generated 3D models and their rendered images in ./logs .

To generate other categories, use the corresponding YAML config file such as vase_demo.yaml . Currently we supprt chair , table , vase , basket , flower and dandelion generators developped by Infinigen .

python ./scripts/sample_diffusion.py --config ./configs/demo/vase_demo.yaml
Training

We train a diffusion model for each procedural generator. The training data is generated by randomly sampling the PCG and render multi-view images. To prepare the training data, run:

python ./scripts/prepare_data.py --generator ChairFactory --save_root /path/to/save/training/data

Replace ChairFactory with other category options as detailed in the ./scripts/prepare_data.py file. This script also conducts offline augmentation and saves the extracted DINOv2 features for each image, which may consume a lot of disk storage. You can adjust the number of the generated data and the render configurations accordingly.

After generating the training data, start the training by:

python ./scripts/train_diffusion.py --config ./configs/train/chair_train.yaml
Use your own PCG

DI-PCG is general for any procedural generator. To train a diffusion model for your PCG, you need to implement the get_params_dict , update_params , spawn_assets , finalize_assets functions and place your PCG in ./core/assets/ . Also change the num_params in your training YAML config file.

If you have any question, feel free to open an issue or contact us.

Citation

If you find our work useful for your research or applications, please cite using this BibTeX:

@article{zhao2024dipcg,
  title={DI-PCG: Diffusion-based Efficient Inverse Procedural Content Generation for High-quality 3D Asset Creation},
  author={Zhao, Wang and Cao, Yanpei and Xu, Jiale and Dong, Yuejiang and Shan, Ying},
  journal={arXiv preprint arxiv:2412.15200},
  year={2024}
}
🤗 Acknowledgements

DI-PCG is built on top of some awesome open-source projects: Infinigen , Fast-DiT . We sincerely thank them all.

Runs of TencentARC DI-PCG on huggingface.co

0
Total runs
0
24-hour runs
0
3-day runs
0
7-day runs
0
30-day runs

More Information About DI-PCG huggingface.co Model

DI-PCG huggingface.co

DI-PCG huggingface.co is an AI model on huggingface.co that provides DI-PCG's model effect (), which can be used instantly with this TencentARC DI-PCG model. huggingface.co supports a free trial of the DI-PCG model, and also provides paid use of the DI-PCG. Support call DI-PCG model through api, including Node.js, Python, http.

TencentARC DI-PCG online free

DI-PCG huggingface.co is an online trial and call api platform, which integrates DI-PCG's modeling effects, including api services, and provides a free online trial of DI-PCG, you can try DI-PCG online for free by clicking the link below.

TencentARC DI-PCG online free url in huggingface.co:

https://huggingface.co/TencentARC/DI-PCG

DI-PCG install

DI-PCG is an open source model from GitHub that offers a free installation service, and any user can find DI-PCG on GitHub to install. At the same time, huggingface.co provides the effect of DI-PCG install, users can directly use DI-PCG installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

DI-PCG install url in huggingface.co:

https://huggingface.co/TencentARC/DI-PCG

Url of DI-PCG

Provider of DI-PCG huggingface.co

TencentARC
ORGANIZATIONS

Other API from TencentARC

huggingface.co

Create photos, paintings and avatars for anyone in any style within seconds.

Total runs: 35.2K
Run Growth: -43.4K
Growth Rate: -124.12%
Updated: July 22 2024
huggingface.co

Total runs: 122
Run Growth: -78
Growth Rate: -55.71%
Updated: December 16 2024
huggingface.co

Total runs: 114
Run Growth: 22
Growth Rate: 19.30%
Updated: November 29 2024
huggingface.co

Total runs: 19
Run Growth: 11
Growth Rate: 57.89%
Updated: December 10 2024
huggingface.co

Total runs: 5
Run Growth: -1
Growth Rate: -20.00%
Updated: December 30 2024
huggingface.co

Total runs: 5
Run Growth: -2
Growth Rate: -40.00%
Updated: December 30 2024
huggingface.co

Total runs: 4
Run Growth: -6
Growth Rate: -150.00%
Updated: December 30 2024
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: June 29 2024
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: August 20 2023
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: December 16 2024
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: August 28 2023
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: December 17 2024
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: August 13 2024
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: April 11 2024
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: October 08 2022
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: January 20 2024