Witness the Rise of AI in Flappy Bird!

Witness the Rise of AI in Flappy Bird!

Table of Contents

1. Introduction
2. Building an AI to Play Flappy Bird
   1. The Simple Game of Flappy Bird
   2. Using Machine Learning in Unity
   3. Training the AI
3. Creating the Game Environment
   1. Designing a Fully Functioning Game
   2. Implementing Player Controls
   3. Scoring System and Difficulty
4. Training the AI with ML Agents
   1. Setting up the Training Environment
   2. Defining the Actions and Observations
   3. Using Ray Perception Sensor for Gathering Information
   4. Adding Manual Observations for Enhanced Learning
5. Implementing Curriculum Learning
   1. Teaching the AI from Easy to Difficult Scenarios
   2. Using Imitation Learning to Bootstrap Training
   3. Adjusting Rewards and Difficulty Levels
6. Analyzing the Training Results
   1. Observing the AI's Performance Over Time
   2. Progressive Adaptation to Changing Environments
   3. Achieving Higher Scores and Perfect Performance
7. Running Training on External Builds
   1. Configuring the Build for ML Agents Training
   2. Running Multiple Instances for Accelerated Training
8. testing and Comparing Human vs AI Performance
   1. Evaluating High Scores and Progression
   2. Demonstrating the AI's Superior Performance
9. Conclusion
10. Resources

Building an AI to Play Flappy Bird

Flappy Bird is a simple yet addictive game that challenges players to navigate a bird through a series of pipes without hitting them. In this article, we will explore the process of building an AI to play Flappy Bird using machine learning in Unity. By training an AI model with the ML Agents toolkit, we aim to surpass human performance and achieve Superhuman gameplay.

The Simple Game of Flappy Bird

Flappy Bird is a straightforward game with one main action: jumping. The objective is to control the bird's flight by tapping the screen to make it flap its wings and avoid colliding with the ground or pipes. As the game progresses, the gaps between pipes become smaller and the pipe heights become more random, making it increasingly challenging.

Using Machine Learning in Unity

To create an AI to play Flappy Bird flawlessly, we leverage the power of ML Agents in Unity. ML Agents provides an intuitive and efficient framework for training intelligent agents using reinforcement learning techniques. The framework allows us to define the actions and observations the AI will utilize to learn and make decisions in the game.

Training the AI

The training process involves progressively teaching the AI how to play Flappy Bird through a curriculum learning approach. We start with basic scenarios and gradually increase the difficulty as the AI becomes more proficient. Initially, we train the AI to avoid only the floor and gradually introduce randomness and smaller gaps between pipes. By fine-tuning rewards and adjusting training parameters, we enable the AI to learn from its own rewards and surpass human performance.

Creating the Game Environment

Before diving into the AI training process, we need to ensure that the game environment is well-designed and functional. In another video, we demonstrate how to create the Flappy Bird game from scratch and set up the basic gameplay mechanics. The game features a single bird controlled by the player, and the score increases as the bird successfully passes through the gaps between pipes.

To Gather information about the game environment, we utilize a ray perception sensor. This sensor emits rays from the bird and detects checkpoints and walls to help the AI identify the position and distance of obstacles. In addition to the ray perception sensor, we also include manual observations such as the bird's Height and the distance to the next pipe. With these observations, the AI has sufficient information to make informed decisions and navigate the game.

Training the AI with ML Agents

Now that we have set up the game environment, we can proceed to train the AI using ML Agents. The first step is to define the actions and observations that the AI will use during training. For Flappy Bird, the AI only needs to perform one action: jumping. We set up a discrete action space with two possible values, representing "jump" and "no jump."

To gather information about the game environment, we utilize the ray perception sensor discussed earlier. This sensor provides valuable data about the location of walls and checkpoints, allowing the AI to navigate through the pipes successfully. Additionally, we include manual observations such as the bird's height and the distance to the next pipe, which provide additional context for decision-making.

Implementing Curriculum Learning

Curriculum learning is a technique that involves gradually increasing the difficulty of training scenarios to help the AI learn more effectively. In Flappy Bird, we manually implement curriculum learning by adjusting the game parameters and rewards as the AI progresses.

Initially, we train the AI in a Scenario with a massive gap and no pipes. The goal is to teach the AI to avoid the top and bottom boundaries of the screen. By utilizing imitation learning with a high strength, the AI learns from demonstrations and starts to avoid collisions. As the AI achieves proficiency in this scenario, we gradually introduce randomness and smaller gaps, challenging the AI to adapt and improve.

Analyzing the Training Results

Throughout the training process, we monitor the AI's performance and track its progress using reward graphs. As the AI encounters new scenarios and adapts to changing environments, the rewards obtained during training fluctuate. However, we observe an overall upward trend, indicating that the AI is learning and improving over time.

The training results reveal the AI's ability to adapt to increased randomness in the game environment. Though initially struggling to cope with the added variation, the AI perseveres and consistently achieves higher rewards. With sufficient training, the AI reaches near-maximum scores, showcasing its superhuman capabilities in playing Flappy Bird.

Running Training on External Builds

To accelerate the training process further, we can run multiple instances of the game on external builds. This approach allows us to utilize all available system resources and speed up the training time. By configuring the build to require no input and running multiple instances simultaneously, we optimize the training process.

Running training on external builds is especially useful for real-world game development scenarios where setting up multiple in-game environments is challenging. By executing the training process on external builds, we can leverage the computing power of multiple instances and achieve faster convergence.

Testing and Comparing Human vs AI Performance

With the training completed, it's time to test the AI's performance and compare it to human gameplay. As a human player, we attempt to achieve high scores by playing the game manually. However, even with our best efforts, the AI consistently outperforms us, surpassing our previous high score.

The final AI demonstrates its superiority by reaching scores well beyond what a human player can achieve. It exhibits exceptional skills and flawlessly navigates through the pipes, demonstrating the potential of machine learning in gaming.

Conclusion

In conclusion, building an AI to play Flappy Bird using machine learning in Unity is a challenging but rewarding endeavor. With the help of ML Agents, we can train an AI to surpass human performance and achieve superhuman gameplay. By implementing curriculum learning, adjusting rewards, and continuously refining the training environment, the AI progressively adapts and becomes more proficient at playing Flappy Bird. With the right training setup and sufficient computational resources, ML Agents can learn to excel in various gaming scenarios.