Train an AI to Master Flappy Bird - Step by Step Guide

Table of Contents

1. Introduction 🌟
2. Training an AI for Flappy Bird
   1. The Simple Design
   2. Setting up the Training Environment
   3. Teaching the AI with Curriculum Learning
   4. Adding Randomness to the Game
3. Training Process and Results
   1. Initial Training with No Pipes
   2. Increasing Difficulty with Random Gap Heights
   3. Making the Gap Sizes Smaller
4. Using a Different Method for Training
   1. Running Training on Normal Builds
   2. Running Multiple Instances for Faster Training
5. testing the Final AI
6. Conclusion

Training an AI to Play Flappy Bird

Flappy Bird is a simple yet challenging game, and the goal here is to train an AI to beat the high score. Using machine learning in Unity, building an AI to play Flappy Bird is actually quite simple. The training process is interesting, starting from teaching the AI to avoid the floor and gradually increasing the difficulty by adding randomness and smaller gaps between pipes. In this article, we will explore the steps involved in training the AI and the results obtained.

The Simple Design

The Flappy Bird game consists of pipes that move from the right side to the left side of the screen. The player-controlled bird needs to navigate through the gaps between the pipes to score points. As time passes, the gaps become smaller and the pipe heights become more random, making the game progressively more difficult. The initial goal is to create an AI that can flawlessly play the game.

Setting up the Training Environment

To train the AI, Unity's ML-Agents toolkit is utilized, which allows for easy integration of machine learning in Unity projects. The AI needs to Gather information about its environment to make decisions. For this, a ray Perception sensor is added to detect checkpoints and walls. The AI also has access to its own positional information, the distance to the next pipe, and the current velocity. With these observations, the AI has sufficient information to accomplish the task.

Teaching the AI with Curriculum Learning

Training an AI to play the game from its inception is challenging, so a curriculum learning approach is adopted. The AI is trained in stages, starting from an easy Scenario and gradually increasing the difficulty. Initially, the AI is trained with no pipes, followed by a scenario with a large gap in the middle. The AI learns to avoid the top and bottom obstacles. Imitation learning, with a high strength value, is used initially to facilitate quick learning from demonstrations.

Adding Randomness to the Game

To make the training more challenging and realistic, randomness is introduced to the game. Randomness is applied to the gap heights between the pipes, making the levels more unpredictable. This step requires the AI to adapt to different gap heights, enhancing its ability to handle varying scenarios. The strength values for imitation learning are reduced, allowing the AI to learn primarily from its own rewards and become better than the demonstrations.

Training Process and Results

Initial Training with No Pipes

In the initial training stage, the AI is trained without any pipes, only a massive gap in the middle. The objective is to teach the AI to avoid the top and bottom obstacles by flying through the gap. By using high strength values for imitation learning, the AI learns mainly from demonstrations. The AI quickly adapts to this scenario and achieves high scores.

Increasing Difficulty with Random Gap Heights

Once the AI becomes proficient in navigating through a gap without pipes, the gap Height is varied randomly. The randomness introduces more unpredictability to the game, making it more challenging for the AI. The strength values for imitation learning are reduced, as the AI learns primarily from its own rewards. The AI initially struggles to adapt to the randomness, but eventually, it improves and achieves higher scores.

Making the Gap Sizes Smaller

As the AI demonstrates proficiency in handling random gap heights, the gap sizes are made smaller, further increasing the difficulty. This step tests the AI's ability to navigate through smaller gaps and sharpens its decision-making skills. The training graph exhibits fluctuating rewards due to the increased randomness in the environment. However, the AI consistently improves and achieves high scores.

Using a Different Method for Training

Running Training on Normal Builds

To overcome the challenge of integrating multiple environments within the game, ML-Agents provides the option to run training on normal builds. By setting up the build to require no input, the game can run without user interaction, facilitating training. This approach eliminates the need for duplicating environments within the game. Models can be trained using multiple instances of the build, significantly speeding up the training process.

Running Multiple Instances for Faster Training

Running multiple instances of the build, each handling a separate training session, accelerates the training process. While resource-intensive, this approach decreases the training time significantly. However, the number of instances depends on the system's resources available. Careful consideration should be given to memory and hard drive space constraints when determining the number of instances.

Testing the Final AI

After completing the training process, it is time to evaluate the AI's performance in playing Flappy Bird. The final AI exhibits remarkable skills, surpassing human capabilities. With its ability to navigate through varying gap heights and sizes, the AI consistently achieves high scores. The results demonstrate that ML-Agents, with the appropriate training setup, can effectively train an AI to master Flappy Bird.

Conclusion

Training an AI to play Flappy Bird using machine learning in Unity is a straightforward process. By gradually increasing the difficulty and incorporating randomness, the AI develops proficiency in navigating through the game's obstacles. ML-Agents' flexibility enables training on normal builds, eliminating the need for complex game modifications. With proper training and environment setup, ML-Agents can be a powerful tool for training AI in various scenarios.