Epic Battle of 1,000 AI Soldiers: Proximal Policy Optimization Shows Astonishing Results

Epic Battle of 1,000 AI Soldiers: Proximal Policy Optimization Shows Astonishing Results

Table of Contents

1. Introduction
2. AI Learning Algorithm: Proximal Policy Optimization (PPO)
3. Musketeers: The Basics of Combat
4. Improving Musketeers' Precision and Friendly Fire Punishment
5. Training Musketeers: Mark II vs Regular Musketeers
6. Warriors: Melee Units with Sprint and Shield Abilities
7. Enhancing Warriors' Battle Tactics
8. Training Warriors: Modified Warriors vs Regular Soldiers
9. Soldiers' Relative Rotation Awareness
10. Final Experiment: Soldiers, Medics, and Rangers Battle

Introduction

In this video, we witness an exciting experiment where 1000 AI soldiers are pitted against each other. These soldiers are trained using a reinforcement learning algorithm called Proximal Policy Optimization (PPO). The AI learns to make strategic decisions by attempting different actions in the environment and being rewarded or punished based on their effectiveness. The goal is for the soldiers to come up with efficient strategies to defeat their opponents. In the following sections, we will explore the enhancements made to the Musketeers, Warriors, and Soldiers, as well as the outcomes of the experiments.

AI Learning Algorithm: Proximal Policy Optimization (PPO)

To enable the AI soldiers to learn how to fight, a reinforcement learning algorithm called Proximal Policy Optimization (PPO) is employed. The AI agents attempt various actions in the environment and are rewarded or punished accordingly. The agents strive to maximize their rewards and minimize punishments, leading them to develop efficient combat strategies.

Musketeers: The Basics of Combat

Musketeers are versatile soldiers capable of moving in any direction and rotating to aim their muskets. They use raycasts, laser-like projections, to detect obstacles and other soldiers. With targeted raycasts, they can accurately shoot enemies within their range. In the first experiment, suggestions from viewers are implemented, focusing on friendly fire punishments and improving precision.

Improving Musketeers' Precision and Friendly Fire Punishment

One of the suggestions implemented is to increase the punishment for friendly fire. By doing so, the Musketeers learn to be more cautious while targeting enemies. Additionally, an alternative to precision aiming is introduced, where Musketeers shoot an additional Raycast in front of them within their shooting distance. This helps them identify enemies and increases the precision of their shots. Punishments for the soldiers' existence are also amplified to motivate them to be more active in combat.

Pros:

• Musketeers become more cautious and accurate in targeting enemies.
• The precision aiming alternative helps improve shooting accuracy.

  Cons:

• Increased punishments for friendly fire may hinder the Musketeers' overall performance.

Training Musketeers: Mark II vs Regular Musketeers

To test the effectiveness of the modifications made to Musketeers, two teams are trained: Team Red consists of regular Musketeers, while Team Blue consists of the enhanced Musketeers (Mark II). Initially, both teams are spawned randomly in a circle. The Musketeers, supported by the reinforcement learning algorithm, gradually learn how to move strategically and engage their opponents.

From the early rounds of training, Team Blue demonstrates initiative and attempts to flank their enemies. However, they struggle to breach enemy lines until around iteration 88, where they successfully eliminate all the red soldiers. With the introduction of spawn points, the Musketeers learn how to advance strategically towards their opponents.

Warriors: Melee Units with Sprint and Shield Abilities

Warriors, unlike Musketeers, are melee units equipped with shields. In addition to movement and rotation capabilities, Warriors can sprint in one direction and use their shields to block attacks. The reinforcement learning algorithm is used to train Warriors to cut distance efficiently and time their attacks effectively.

Enhancing Warriors' Battle Tactics

A suggestion from a viewer is incorporated into the training process: Warriors are rewarded for being close to enemies and punished for being far from them. This modification aims to encourage Warriors to engage closely with opponents, maximizing the effectiveness of their attacks.

Pros:

• Warriors become more skilled in closing distances and striking opponents.

  Cons:

• The modification may require fine-tuning to balance rewards and punishments effectively.

Training Warriors: Modified Warriors vs Regular Soldiers

To compare the performance of modified Warriors with regular soldiers, two teams are trained: Team Red consists of the enhanced Warriors, while Team Blue consists of regular soldiers. Both teams undergo training involving simulated combat scenarios.

In the final battle between the two teams, the modified Warriors (Team Red) showcase dominance by pushing forward aggressively. Despite the regular soldiers' brave attempts at defending, they are gradually overwhelmed by the relentless assault of Team Red. Eventually, only one soldier from the losing team survives before meeting a heroic demise.

Soldiers' Relative Rotation Awareness

In the next experiment, soldiers are equipped with the ability to assess the rotation of the soldier in front of them. This relative rotation awareness enables them to capitalize on advantageous situations, such as when an enemy is unaware of their presence.

Final Experiment: Soldiers, Medics, and Rangers Battle

In the final experiment, 1000 soldiers are trained, consisting of 20 Medics, 40 Warriors, and 40 Rangers in each team. Notably, Medics are rewarded for being close to allies and punished for being close to enemies.

The battle commences with fierce engagement from both teams. Soldiers from each side clash in a struggle for territory. Tactics such as flanking and teamwork are observed as soldiers fight with all their might. Blue team seems to gain an advantage initially, but eventually succumbs to the overwhelming forces of the red team. The battle ends with only one red soldier remaining - a lost Medic with no means to heal or defeat the enemy.

Conclusion

Overall, the experiments prove to be a remarkable success, with many of the enhancements improving the agents' combat abilities. Strategies such as adjusting punishments, increasing precision, and rewarding proximity to enemies enhance the soldiers' performance. While not every modification yields the desired outcome, overall training results in efficient and skillful soldiers. Future experiments may explore the soldiers' capabilities in handling fast zombies or other challenging scenarios.

Highlights:

• 1000 AI soldiers engaged in intense battles
• Musketeers gained precision and faced harsher punishments for friendly fire
• Training outcomes of Mark II Musketeers and regular Musketeers
• Warriors employed sprinting and shielding tactics
• Comparison between modified Warriors and regular soldiers
• Soldiers gained relative rotation awareness
• Final battle involving Medics, Warriors, and Rangers
• Successful and unsuccessful enhancements to soldiers' combat abilities

FAQ:

Q: What reinforcement learning algorithm is used to train the AI soldiers? A: The algorithm used is called Proximal Policy Optimization (PPO).

Q: How are Musketeers trained to become more precise and avoid friendly fire? A: Friendly fire punishments are increased, and Musketeers are equipped with an alternative to precision aiming through additional raycasts.

Q: What is the outcome of the battles between modified and regular soldiers? A: In general, the modified soldiers exhibited improved performance, but not all modifications yielded successful results.

Q: Do Warriors possess any unique abilities? A: Warriors have the ability to sprint in one direction and use their shields to block attacks.

Q: Are there any modifications made to Warriors' combat abilities? A: Yes, Warriors are rewarded for being close to enemies and punished for being far from them to encourage close engagement.

Q: What abilities do Medics, Warriors, and Rangers have in the final battle? A: Medics focus on healing, and Warriors and Rangers specialize in melee and ranged combat, respectively.

Q: Was the training of AI soldiers successful overall? A: Yes, the experiments yielded impressive results, with soldiers demonstrating improved combat strategies and skills.