Training an AI to Beat Human Racers in Gran Turismo

Table of Contents

1. Introduction
2. Moving to Tokyo for an Exciting Internship
3. Training an AI to Race in Gran Turismo
4. Challenges Faced during the Pandemic
5. Sony AI: Unleashing Human Imagination with AI Technology
6. Building a Global Organization
7. Scaling the Project: Can AI Beat the Best Gran Turismo Drivers?
8. The Role of Florian in Setting the Foundation
9. Setting Ambitious Goals: Beating the World's Best Gran Turismo Racers
10. Understanding Intelligence: Creating an Intelligent System
11. Collaborating with Polyphony Digital: The Makers of Gran Turismo
12. The Realism and Difficulty of Gran Turismo
13. Reinforcement Learning and Training Sophie
14. Teaching Sophie to Drive: Learning to Avoid Walls and Control Speed
15. Going Beyond Mimicking: Training Sophie to Beat the Best Drivers
16. AI Handling Unexpected Situations on the Racetrack
17. The Technical Leadership of Pete Wurman
18. Team Organization and Focus Areas
19. Evaluating GT Sophie's Performance Against Human Drivers
20. Overcoming Challenges and Facing Uncertainty on the Course
21. Conclusion

🏎️ Training an AI to Race in Gran Turismo

In the beginning of 2020, I moved to Tokyo, Japan for a very exciting internship opportunity with Sony. Florian, a fellow intern, started working just beside me. Our goal was to train an AI to race around the Gran Turismo track faster than any human. However, our plans took an unexpected turn when the pandemic hit.

Moving to Tokyo for an Exciting Internship

The opportunity to work with Sony in Tokyo was something I couldn't resist. The chance to be part of the cutting-edge world of artificial intelligence and gaming was a dream come true. Little did I know the challenges that lay ahead.

Challenges Faced during the Pandemic

As the pandemic struck, I had to take all of my PlayStations home to my tiny apartment. The AI training required running multiple simulations of Gran Turismo Sport, distributed over nine PlayStations. This meant that I had to keep a close eye on them, ensuring they were still running and generating the desired results.

The PlayStations ran 24/7, generating a lot of heat and noise. Initially, I didn't worry about the electricity bill since it was all done by Sony. However, I received a shocking email that informed me the electricity bill had increased sixfold. It made me wonder what kind of strange things were going on in my apartment.

Sony AI: Unleashing Human Imagination with AI Technology

Sony AI, headed by CEO Michael Spranger, was established with the purpose of unleashing human imagination and creativity through new AI technology. Their primary focus is on AI as a tool for human interaction, whether it be for creators, gamers, or individuals utilizing their technology.

The organization is built on ambition, attracting smart individuals who want to work on interesting and amazing projects. With offices in Tokyo, Zurich, and a remote workforce in North America, Sony AI aims to create an environment that fosters innovation and attracts the best talent from around the world.

Scaling the Project: Can AI Beat the Best Gran Turismo Drivers?

Florian's initial experiments laid the foundation for our project, sparking Curiosity about whether we could build an AI capable of outperforming the best Gran Turismo drivers. In the fall of 2020, we embarked on the project in a serious manner, setting a deadline to beat the world's best Gran Turismo racers.

At first, I thought the milestones set for reaching this goal were too ambitious. But as we delved deeper into the project, the concept of intelligence became more intriguing. We aimed to create a system that possessed intelligence similar to humans, with the ability to make decisions and perform complex tasks.

Collaborating with Polyphony Digital: The Makers of Gran Turismo

To truly understand the dynamics and challenges of racing in Gran Turismo, we needed to collaborate with the experts. Polyphony Digital, the Game developing Studio behind the Gran Turismo franchise, provided valuable insights. Their racing simulation offered unrivaled realism, making it incredibly difficult to achieve perfect control on the track.

The professionals and expert gamers who excelled in Gran Turismo were the epitome of skill in a highly challenging task. We sought advice from Peter Stone, one of the most prominent reinforcement learning researchers, to ensure our project pushed the boundaries of AI development.

Reinforcement Learning and Training Sophie

Our training process involved a trial and error approach. Sophie, our AI agent, began by behaving randomly, just as a beginner driver would. She had to learn the basics, such as driving in the right direction and avoiding walls and obstacles that would slow her down.

We used the same principles used to train animals, rewarding Sophie for making the right decisions and encouraging her to perform complex tasks. The goal was not to mimic the best human drivers, but to surpass them. Sophie was designed to train herself, gradually improving her racing skills with each trial.

AI Handling Unexpected Situations on the Racetrack

As the training progressed, we realized that racing fast around the track was just one aspect of the challenge. Dealing with other cars and the unexpected nature of actions taken by human drivers posed a greater challenge. It required the AI to adapt to various scenarios, making split-Second decisions and anticipating movements.

The project's technical lead, Pete Wurman, played a crucial role in guiding the team through the intricate decisions and research required to overcome these challenges. The teams were organized to focus on different aspects, including time trial performance, racing against other drivers, Sophie's internal architecture, and the rules of partnership.

Evaluating GT Sophie's Performance Against Human Drivers

One of the greatest uncertainties we faced was how well GT Sophie would perform against human drivers. Humans behave differently on the racetrack compared to AI agents. Watching GT Sophie compete in actual races was stressful, as it was always on the verge of a massive accident.

Despite the initial setbacks, we remained committed to our goal. We had to overcome the difficulties and fine-tune Sophie's abilities to race competitively against human opponents. The project was much more challenging than we initially anticipated, pushing us to explore the limits of AI technology.

Conclusion

Training an AI to race in Gran Turismo was a rollercoaster journey filled with unexpected obstacles and remarkable achievements. Sony's dedication to AI technology and collaboration with industry experts played a vital role in the project's success. The ability of GT Sophie to compete with and outperform human drivers demonstrated the immense potential of AI in the world of gaming and beyond.

Highlights:

• Turning an internship project into a groundbreaking AI racing project
• Overcoming challenges and scaling the project during the pandemic
• Sony AI's vision to unleash human imagination with AI technology
• Collaborating with Polyphony Digital to understand the realism and difficulty of Gran Turismo
• Reinforcement learning and training Sophie to beat the best drivers
• GT Sophie's ability to handle unexpected situations on the racetrack
• Evaluating GT Sophie's performance against human drivers

FAQs

Q: How did the pandemic affect the AI racing project? A: The pandemic forced the interns to work remotely and move the project to their small apartments. This unexpected shift brought about challenges in managing the PlayStations, increased electricity bills, and uncertainty regarding the project's progress.

Q: What were the key goals of Sony AI? A: Sony AI aimed to create an ambitious global organization that attracted the best talent from around the world. They wanted to build an environment that fostered innovation, allowing AI technology to be utilized as a tool for interaction between humans and technology, whether it be in gaming, creativity, or other domains.

Q: How did the collaboration with Polyphony Digital contribute to the project? A: Collaborating with Polyphony Digital, the makers of Gran Turismo, provided essential insights into the realistic dynamics of the game. It allowed the AI project team to understand the challenges faced by expert human drivers and mimic those challenges in training Sophie to race competitively.

Q: What obstacles did the AI face when racing against human drivers? A: Racing against human drivers required the AI to adapt to unexpected situations and anticipate the actions of other drivers. This posed a greater challenge compared to racing alone on the track. The AI had to master the art of strategic decision-making, finding alternate paths, and utilizing slipstreams to pass opponents.

Q: How did reinforcement learning help in training GT Sophie? A: Reinforcement learning allowed GT Sophie to learn from trial and error. Initially behaving randomly, the AI was gradually rewarded for making correct decisions and encouraged to perform complex tasks. This self-training approach enabled GT Sophie to outperform human drivers through constant improvement and adaptation.

Resources:

• Sony AI
• Polyphony Digital