Unleashing the Power of AI: Conquering the Hockolicious Track in Trackmania

Table of Contents

1. Introduction
2. Training the AI
   • Learning the map and Game
   • Reinforcement learning
   • Value-based reinforcement learning
3. Challenges Faced by the AI
   • The first hurdle: jumping over a barrier
   • Navigating an elevated road without walls
   • Pillars in the middle of the road
   • Small step on the road
4. Progress and Achievements
   • Improving race time
   • Beating benchmarks and world records
5. Conclusion
6. Future Improvements
7. The AI's Best Run
8. Interpreting the value bar
9. Comparison with Human Players
10. Next Challenges

🏎️ Teaching an AI to Conquer the Prestigious Trackmania Course 🏆

In this video, we embark on an exciting challenge to train our very own AI to master the highly prestigious track, Hockolicious, in the popular racing game Trackmania. While our AI has shown promising results in simpler tracks, conquering Hockolicious presents a whole new level of difficulty. Join us as we push the limits of AI learning and witness the progress we make in exceeding expectations.

1. Introduction

Trackmania is a game that requires not only skill but also an understanding of the track layout and the ability to make split-Second decisions. Teaching an AI to play such a complex game is no easy feat. However, we are determined to train our AI from scratch, meaning it needs to learn not just the map, but also the fundamental mechanics of the game itself. This entails training it to race through repeated interactions, learning from its successes and failures.

2. Training the AI

- Learning the map and game

To begin training our AI, it must familiarize itself not only with the map but also with the game mechanics. Imagine an AI that has never seen a racing game or even a car before. Through trial and error over thousands of races, the AI gradually learns to navigate the track and understand the game's dynamics.

- Reinforcement learning

To guide the AI's learning process, the game provides rewards or punishments based on its performance in each race. The AI then aims to maximize its reward by using a neural network to predict the expected reward for each action it can take. This form of learning, known as reinforcement learning, ensures that the AI continuously refines its strategies to achieve better outcomes.

- Value-based reinforcement learning

The AI's decision-making process is further enhanced through value-based reinforcement learning. By presenting the AI with multiple combinations of keypresses, it can choose the action that offers the highest expected reward. For example, the AI may need to decide between going left, forward, or right. Through accurate predictions, the AI makes informed choices, maximizing its chances of success.

3. Challenges Faced by the AI

Conquering Hockolicious is not a straightforward task. The track presents numerous hurdles that even skilled human players find challenging. Our AI must overcome these obstacles through its learning process.

- The first hurdle: jumping over a barrier

In its early stages of training, the AI encounters a significant hurdle. It must learn to make a precise jump over a barrier to continue racing. Initially, the AI struggles with the necessary speed and accuracy, often getting stuck and unable to progress. However, as training progresses, the AI gradually masters the jump and moves forward.

- Navigating an elevated road without walls

The track also includes a section with an elevated road that lacks protective walls. Any mistake in navigation here results in the AI falling from the track with no hope of recovering. Learning to navigate this section safely pushes the AI to become even more proficient in its decision-making abilities.

- Pillars in the middle of the road

As if that wasn't challenging enough, the AI encounters an unexpected obstacle — pillars placed right in the middle of the road. Previous strategies of staying away from the walls no longer work. The AI must adapt its approach and find new ways to navigate through this maze of pillars.

- Small step on the road

Approaching the finish line, the AI faces another obstacle that may go unnoticed by human players. There is a small step on the road that requires sufficient speed to overcome. At this point, the AI is still in the early stages of learning to drive and will explore random moves, making it unlikely to successfully pass this obstacle.

4. Progress and Achievements

Throughout the training process, the AI steadily improves its race time and performance. With each iteration, it becomes more skilled at drifting, taking tighter lines, and optimizing its trajectories. Let's take a moment to observe the AI's progress and achievements.

- Improving race time

After approximately two hours of training, the AI starts to consistently complete races. We can now measure its race time and observe its progress on a curve graph. With time, the AI's race time improves as it acquires and applies more advanced racing strategies.

- Beating benchmarks and world records

The AI's development is not only evident through improved race times but also by surpassing established benchmarks. It surpasses the game's gold medal time, as well as the time set by the authors of this video. This achievement is notable, considering even veteran human players would struggle to achieve the same level of performance even with extensive practice.

5. Conclusion

In this challenging endeavor, our AI has proven to be an incredible competitor. Its ability to learn and adapt to the complexities of Trackmania showcases the power of AI and its potential for pushing boundaries. We are immensely proud of the progress we have made, but this is just the beginning.

6. Future Improvements

While we have already exceeded expectations, there is always room for improvement. We will continue to explore algorithmic enhancements to further strengthen our AI's skills. One potential area for improvement is increasing the frame rate for more precise gameplay.

7. The AI's Best Run

Finally, we invite you to witness the impressive culmination of our AI's training. Its best run showcases the AI's enhanced confidence in predicting future rewards. The AI's estimation of rewards to come is vital in planning its strategies, even if it means occasionally slowing down to ensure a successful outcome.

8. Interpreting the value bar

To better understand the AI's decision-making process, it is essential to grasp how the agent's reward is defined. The value bar on the screen represents the total distance the car is expected to travel in the next seven seconds. This knowledge helps interpret instances where the value bar decreases despite the car's speed increasing, as the AI considers upcoming turns and adjusts its speed accordingly.

9. Comparison with Human Players

To put the AI's achievements into perspective, we have compared its performance to that of human players. Out of all the players who have attempted the track in the past fifteen years, only 23 have managed to achieve better times than our AI. This further highlights the AI's exceptional progress and competence.

10. Next Challenges

Having conquered Hockolicious and surpassed expectations, we now turn to you, our viewers, for suggestions on the next challenges our AI should undertake. We eagerly await your input and can't wait to continue pushing the limits of AI gaming excellence.

Thank you for joining us on this exhilarating journey, and we look forward to bringing you more exciting AI adventures in the future.

Highlights:

1. Training an AI to conquer Trackmania's prestigious track, Hockolicious
2. AI learning through trial and error in map and game mechanics
3. Reinforcement learning and value-based reinforcement learning
4. Overcoming challenges: jumping barriers, navigating without walls, and more
5. Progress and achievements: improving race times and surpassing benchmarks
6. Future improvements and the AI's best run showcased
7. Interpreting the AI's value bar and comparison with human players
8. Seeking suggestions for the AI's next challenges.