Unlock the Power: Stable Diffusion on Apple Silicon

Table of Contents

1. Introduction
2. Installing Mini Conda
3. Cloning the GitHub Repository
4. Creating a Virtual Environment
5. Installing the Converter Requirements
6. Using Hugging Face for Machine Learning Models
7. Selecting and Downloading the Model
8. Converting the Model to Core ML
9. Using the Converted Model
10. Conclusion

Introduction

In this article, we will explore how to run Stable Diffusion models on Apple silicon by converting them to core ml. We will start by installing Mini Conda, a popular software that allows the creation of virtual environments to manage dependencies and Python versions. Next, we will clone the GitHub repository from Apple and Create a virtual environment using Mini Conda. Once the environment is set up, we will install the requirements for the converter. Then, we will explore Hugging Face, a platform for machine learning models and data sets, and select the model we want to download and convert. After accepting the terms and conditions, we will ask the module to convert the model to core ml. Finally, we will learn how to use the converted model for generating images. So, let's get started!

Installing Mini Conda

To run stable diffusion models on Apple silicon, we need to install Mini Conda. Mini Conda is a Python virtual environment that allows us to manage dependencies and Python versions independently between projects. Start by downloading the Mini Conda version suitable for your system from the official Website. Once the download is complete, open your terminal and navigate to the downloads folder. Run the command bash mini_conda.sh to initiate the installation process. Follow the Prompts and agree to the terms and conditions. Once the installation is complete, You can move on to the next step.

Cloning the GitHub Repository

Now that Mini Conda is installed, let's clone the GitHub repository from Apple. Create a new folder for your projects and navigate to it in the terminal. Execute the command git clone <repository-url> to clone the repository onto your local machine. Once the clone process is finished, navigate into the repository folder.

Creating a Virtual Environment

Before we can proceed with the conversion, we need to create a virtual environment using Mini Conda. In the terminal, run the command conda create -n core_ml_stable_diffusion python=<python-version> to create a new virtual environment named "core_ml_stable_diffusion". Replace <python-version> with the desired version of Python. After specifying the Python version, confirm the installation by entering 'y' when prompted. The virtual environment will be created, and you can proceed to activate it.

Installing the Converter Requirements

To successfully convert the model to core ml, we need to install the requirements. In the terminal, make sure you are in the repository folder and activate the virtual environment by running the command conda activate core_ml_stable_diffusion. This will switch your Python environment to the newly created virtual environment. Once activated, run the command pip install -r requirements.txt to install the necessary packages for the converter. This process may take a few minutes as multiple dependencies are being installed.

Using Hugging Face for Machine Learning Models

Now, let's explore Hugging Face, a platform that provides machine learning models and data sets. If you don't have an account, create one on the Hugging Face website. Once logged in, go to your account settings and create an access token. This token will allow you to download models from Hugging Face using the command-line interface.

Selecting and Downloading the Model

Choose the version of the model you want to download and convert. Click on the corresponding link in the repository to access the model's page. If this is your first time visiting the model, you will be prompted to accept the terms and conditions. After accepting, you can proceed to download the model. Make sure to copy the download URL.

Converting the Model to Core ML

With the model downloaded, it's time to convert it to core ml. In the terminal, use the following command:

python -m coreml_stable_diffusion.torch_to_coreml \
    convert_text_encoder \
    convert_vae_decoder \
    convert_safety_checker \
    --model-version <model-version> \
    --output-folder <output-folder>

Replace <model-version> with the version number of the model you downloaded, and specify the desired output folder name as <output-folder>. The conversion process may take some time depending on the size of the model.

Using the Converted Model

Once the conversion is complete, you can start using the converted model to generate images. In the terminal, run the command:

python -m coreml_stable_diffusion.pipeline \
    --prompt "<prompt-description>" \
    -i <converter-model-folder> \
    -o <output-folder> \
    -c <compute-unit> \
    -s <seed> \
    -m <model-version>

Replace <prompt-description> with the desired description for the image you want to generate. Specify the folder containing the converted model as <converter-model-folder>. Choose an output folder name as <output-folder>. Select the desired compute unit by replacing <compute-unit> with either "CPU", "NE", or "GPU". Set the <seed> value as desired. Finally, specify the model version using <model-version>. The image generation process will begin, and the output will be saved in the specified output folder.

Conclusion

In this article, we discussed how to run stable diffusion models on Apple silicon by converting them to core ml. We installed Mini Conda, cloned the GitHub repository from Apple, created a virtual environment, and installed the converter requirements. We also explored Hugging Face for machine learning models and downloaded the desired model. Finally, we converted the model to core ml and used the converted model for image generation. With this knowledge, you can now leverage stable diffusion models on Apple silicon for your own projects. Happy coding!

Highlights:

• Learn how to run stable diffusion models on Apple silicon
• Install Mini Conda and create virtual environments
• Convert machine learning models to core ml
• Use Hugging Face for model selection and downloading
• Generate images using the converted model

FAQ: Q: Can I use Mini Conda with other Python projects? A: Yes, Mini Conda allows you to create independent virtual environments for different projects, enabling better management of dependencies.

Q: Are there different versions of stable diffusion models available? A: Yes, you can select the desired version from the available options in the GitHub repository.

Q: What are the supported compute units for image generation? A: You can choose between CPU, NE (Neural Engine), and GPU, depending on your device capabilities and requirements.