TencentARC / ColorFlow

huggingface.co
Total runs: 0
24-hour runs: 0
7-day runs: 0
30-day runs: 0
Model's Last Updated: January 12 2025

Introduction of ColorFlow

Model Details of ColorFlow

🎨 ColorFlow

Retrieval-Augmented Image Sequence Colorization

Authors: Junhao Zhuang, Xuan Ju, Zhaoyang Zhang, Yong Liu, Shiyi Zhang, Chun Yuan, Ying Shan

Your star means a lot for us to develop this project! :star:

🌟 Abstract

Automatic black-and-white image sequence colorization while preserving character and object identity (ID) is a complex task with significant market demand, such as in cartoon or comic series colorization. Despite advancements in visual colorization using large-scale generative models like diffusion models, challenges with controllability and identity consistency persist, making current solutions unsuitable for industrial application.

To address this, we propose ColorFlow , a three-stage diffusion-based framework tailored for image sequence colorization in industrial applications. Unlike existing methods that require per-ID finetuning or explicit ID embedding extraction, we propose a novel robust and generalizable Retrieval Augmented Colorization pipeline for colorizing images with relevant color references.

Our pipeline also features a dual-branch design: one branch for color identity extraction and the other for colorization, leveraging the strengths of diffusion models. We utilize the self-attention mechanism in diffusion models for strong in-context learning and color identity matching.

To evaluate our model, we introduce ColorFlow-Bench , a comprehensive benchmark for reference-based colorization. Results show that ColorFlow outperforms existing models across multiple metrics, setting a new standard in sequential image colorization and potentially benefiting the art industry.

🚀 Getting Started

Follow these steps to set up and run ColorFlow on your local machine:

  • Clone the Repository

    Download the code from our GitHub repository:

    git clone https://github.com/TencentARC/ColorFlow
    cd ColorFlow
    
  • Set Up the Python Environment

    Ensure you have Anaconda or Miniconda installed, then create and activate a Python environment and install required dependencies:

    conda create -n colorflow python=3.8.5
    conda activate colorflow
    pip install -r requirements.txt
    
  • Run the Application

    You can launch the Gradio interface for PowerPaint by running the following command:

    python app.py
    
  • Access ColorFlow in Your Browser

    Open your browser and go to http://localhost:7860 . If you're running the app on a remote server, replace localhost with your server's IP address or domain name. To use a custom port, update the server_port parameter in the demo.launch() function of app.py.

🎉 Demo

You can try the demo of ColorFlow on Hugging Face Space.

🛠️ Method

The overview of ColorFlow. This figure presents the three primary components of our framework: the Retrieval-Augmented Pipeline (RAP) , the In-context Colorization Pipeline (ICP) , and the Guided Super-Resolution Pipeline (GSRP) . Each component is essential for maintaining the color identity of instances across black-and-white image sequences while ensuring high-quality colorization.

🤗 We welcome your feedback, questions, or collaboration opportunities. Thank you for trying ColorFlow!

📰 News
  • Release Date: 2024.12.17 - Inference code and model weights have been released! 🎉
📋 TODO
  • ✅ Release inference code and model weights
  • ⬜️ Release training code
📜 Citation
@article{zhuang2024colorflow,
    title={ColorFlow: Retrieval-Augmented Image Sequence Colorization},
    author={Zhuang, Junhao and Ju, Xuan and Zhang, Zhaoyang and Liu, Yong and Zhang, Shiyi and Yuan, Chun and Shan, Ying},
    journal={},
    year={2024}
}
📄 License

Please refer to our license file for more details.

Runs of TencentARC ColorFlow on huggingface.co

0
Total runs
0
24-hour runs
0
3-day runs
0
7-day runs
0
30-day runs

More Information About ColorFlow huggingface.co Model

ColorFlow huggingface.co

ColorFlow huggingface.co is an AI model on huggingface.co that provides ColorFlow's model effect (), which can be used instantly with this TencentARC ColorFlow model. huggingface.co supports a free trial of the ColorFlow model, and also provides paid use of the ColorFlow. Support call ColorFlow model through api, including Node.js, Python, http.

TencentARC ColorFlow online free

ColorFlow huggingface.co is an online trial and call api platform, which integrates ColorFlow's modeling effects, including api services, and provides a free online trial of ColorFlow, you can try ColorFlow online for free by clicking the link below.

TencentARC ColorFlow online free url in huggingface.co:

https://huggingface.co/TencentARC/ColorFlow

ColorFlow install

ColorFlow is an open source model from GitHub that offers a free installation service, and any user can find ColorFlow on GitHub to install. At the same time, huggingface.co provides the effect of ColorFlow install, users can directly use ColorFlow installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

ColorFlow install url in huggingface.co:

https://huggingface.co/TencentARC/ColorFlow

Url of ColorFlow

Provider of ColorFlow huggingface.co

TencentARC
ORGANIZATIONS

Other API from TencentARC

huggingface.co

Create photos, paintings and avatars for anyone in any style within seconds.

Total runs: 35.2K
Run Growth: -43.4K
Growth Rate: -124.12%
Updated: July 22 2024
huggingface.co

Total runs: 122
Run Growth: -78
Growth Rate: -55.71%
Updated: December 16 2024
huggingface.co

Total runs: 114
Run Growth: 22
Growth Rate: 19.30%
Updated: November 29 2024
huggingface.co

Total runs: 19
Run Growth: 11
Growth Rate: 57.89%
Updated: December 10 2024
huggingface.co

Total runs: 5
Run Growth: -1
Growth Rate: -20.00%
Updated: December 30 2024
huggingface.co

Total runs: 5
Run Growth: -2
Growth Rate: -40.00%
Updated: December 30 2024
huggingface.co

Total runs: 4
Run Growth: -6
Growth Rate: -150.00%
Updated: December 30 2024
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: June 29 2024
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: August 20 2023
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: December 16 2024
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: December 20 2024
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: August 28 2023
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: December 17 2024
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: August 13 2024
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: April 11 2024
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: October 08 2022
huggingface.co

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated: January 20 2024