Stable Diffusion教學

Table of Contents

1. Introduction
2. Training Lora
   • Stable Diffusion and Checkpoint Models
   • The Role of Lora Model
   • Tips for Training
3. Preparing Training Data
   • Image Resolution
   • Obtaining Images
   • Quantity and Variety of Images
   • Modifying Image Resolution
   • Generating Prompt Text
   • Editing Prompt Texts
4. Installing Kohya_ss GUI
   • Required Dependencies
   • Visual Studio Installation
   • Installing Kohya_ss GUI
   • CUDNN 8.6 Installation (Optional)
5. Configuring Kohya_ss GUI
   • Choosing Configuration File
   • Selecting Source Model
   • Specifying Folders
   • Setting Training Parameters
   • Initiating Model Training
   • Checking Lora Status
6. Testing Lora Model
   • Usage Instructions
   • Adjusting Pose and Clothing
   • Dealing with Overlapping Effects
   • Uploading and Sharing Models
7. Conclusion
8. Next Steps: ControlNet Plugin

Training Lora

Stable diffusion is an advanced technique for generating high-quality images using checkpoint models. In the Context of stable diffusion, the role of the Lora model is crucial. The Lora model acts as a controller, allowing You to control the characteristics and features of the generated images. However, it is important to note that the Lora model's behavior can be unpredictable at times, similar to an artist deviating from the given instructions. Therefore, it may require multiple attempts to achieve the desired results.

Tips for Training Lora

To effectively train the Lora model, it is essential to prepare a sufficient amount of training data. The resolution of the images used for training plays a significant role in the quality of the generated images. While the mainstream resolution is 512x512, training a 768x768 model can improve the image quality. However, it is important to note that training a higher-resolution model requires more training time and a larger VRAM.

When selecting images for training, it is recommended to include a variety of angles, expressions, and lighting conditions. The more detailed and diverse the training data, the better the quality of the generated images. You can obtain images from animations, games, movies, or Create your own using tools like Daz Studio or Character Creator. Additionally, AI drawing tools can also be used to generate training images.

It is recommended to have a minimum of 15 images for training the Lora model, with each image being trained for a minimum of 100 steps. This ensures a total training step count of at least 1500 steps. The more images you have for training, the better the results. It is important to note that if you have fewer than 15 images, you can adjust the training steps accordingly to achieve a total step count of 1500.

Installing Kohya_ss GUI

To begin the training process, you will need to install the Kohya_ss GUI. The installation process requires several dependencies, including Python 3.10, Git, and Visual Studio. Once the dependencies are installed, you can proceed with the installation of the Kohya_ss GUI. It is recommended to follow the installation instructions provided by the developer to ensure a successful installation.

If you have a graphics card with 8GB or lower VRAM, it is recommended to enable the "Memory efficient Attention" option during the configuration process. This option reduces VRAM usage during training, preventing potential failures due to VRAM overflow. However, please note that enabling this option will result in slower training speed.

Configuring Kohya_ss GUI

Before training the Lora model, you need to configure the settings in the Kohya_ss GUI. This includes selecting the appropriate configuration file, specifying the source model, and setting the folder paths for images, logs, and model outputs. Additionally, you can fine-tune training parameters such as batch size and maximum resolution Based on your specific requirements.

Once the configuration is complete, you can initiate the model training process. Please note that the training time depends on various factors such as the speed of your graphics card, the resolution of the model, the desired number of training steps, and the quantity and quality of the training images. It is recommended to monitor the GPU temperature during training to ensure optimal performance.

Testing Lora Model

After the training process is completed, you can test the Lora model by generating sample images. By inputting the appropriate prompt texts and Lora weights, you can control the characteristics of the generated images. It is important to note that when using 3D render-based reference images with checkpoint models or photos, you may encounter overlapping effects. To minimize this issue, you can adjust the "CFG Scale" parameter and add negative prompt texts like "3D". This helps reduce the smoothness of the character's appearance.

Once you are satisfied with the trained Lora model, you can upload and share it with others. There are platforms like Civit AI where you can share your models with the community. Sharing your models allows others to download and utilize them for their projects.

Conclusion

Training the Lora model requires attention to Detail and patience. By following the steps outlined in this guide, you can successfully train a personalized AI model that meets your specific requirements. Remember to experiment with different training data, settings, and prompt texts to achieve the desired results.