Unlocking the Potential: Evolutionary AI in Age of Empires II

Table of Contents

1. Introduction
2. Transition to AOC
3. Issues with AI Evolution in DE
4. Improved Training Using AOC
5. Initial AI Behaviors in AOC
6. The Importance of Villager Production in Early Game
7. Future Improvements for Villager Management
8. Progression Towards Feudal Age
9. Optimizations and Sub-optimal Behaviors
10. The Emergence of Army Training
11. Inconsistencies in Luring Boars
12. The Aesthetics of Genetic Algorithms
13. The Potential of Emergent Behaviors
14. Expanding AI Development
15. The Challenge of Implementing Resign Scripts

Introduction

In this project, we are exploring the capabilities of genetic algorithm-based Artificial Intelligences (AIs) in the game "Age of Empires II". We have developed an AI system that utilizes the original Age of Conquerors (AOC) version of the game for quicker training. While there have been some challenges in porting AI from User Patch 1.5 to Definitive Edition (DE), we are focused on training and improving our AIs in AOC for now.

Transition to AOC

To enhance the training process, we have shifted from using DE to AOC. This change allows us to run multiple instances of the game, speeding up the training process significantly. However, it should be noted that the AIs developed in AOC may not directly transfer to DE without modifications.

Issues with AI Evolution in DE

One of the issues we encountered while training AIs in DE was the evolution of AIs reaching the Feudal Age. This unexpected behavior was a result of starting the game on higher difficulty levels, which provided bonus resources to the AIs. As a result, the AIs did not prioritize resource collection and focused solely on advancing to the next age. We have yet to find a solution to this issue and will continue to explore ways to overcome it.

Improved Training Using AOC

Despite the challenges Mentioned earlier, training AIs in AOC has proven to be more efficient. Our AIs have already shown promising behaviors, such as creating lumber camps and consistently producing villagers. These early-game strategies are vital for resource collection and establishing a strong foundation.

Initial AI Behaviors in AOC

In the initial stages of training, our AIs prioritize putting villagers on food and maintaining a constant flow of villagers. This strategy is essential for the early game and ensures a steady supply of resources. As the training progresses and the AI becomes more complex, we anticipate it will develop strategies to optimize villager production and resource management.

The Importance of Villager Production in Early Game

Villager production is a critical aspect of early-game strategy in Age of Empires II. Our AI has recognized this importance and consistently focuses on producing villagers. This strategy sets a strong foundation for resource collection and sets the stage for future developments in the game.

Future Improvements for Villager Management

While consistent villager production is crucial in the early game, it may become beneficial for our AI to limit the number of villagers it creates as the game progresses. Balancing villager production with other strategic considerations will be a key area of improvement for our AI.

Progression Towards Feudal Age

In our current training phase, our AI has not yet shown the ability to advance to the Feudal Age. However, we anticipate that as we make further modifications to our scoring system and reward system, the AI will learn to prioritize and successfully progress to the next age.

Optimizations and Sub-optimal Behaviors

During the learning process, our AI has displayed both optimized behaviors and sub-optimal tendencies. For example, the AI has learned to build multiple barracks and consistently produce soldiers. However, it has also displayed inconsistencies in luring boars, an action that provides an advantage in the game. These inconsistencies will be addressed and refined as the AI evolves.

The Emergence of Army Training

A significant breakthrough in our AI's development is its ability to train and employ armies. This emergence of the AI's capacity to strategize and deploy military forces is an exciting development. While it is still refining its tactics and deciding when to initiate battles, its progress is promising.

Inconsistencies in Luring Boars

The AI has shown intermittent success in luring boars, which provides valuable food resources. Although the AI's behavior around boars lacks consistency, it has still managed to benefit from this tactic. Fine-tuning this behavior and ensuring its consistent implementation will be a focus for future development.

The Aesthetics of Genetic Algorithms

As someone who has worked with genetic algorithms for a significant amount of time, observing the development of AI through this method have been awe-inspiring. While the early stages may appear chaotic and unrefined, witnessing the emergence of behaviors without human guidance is a fascinating journey.

The Potential of Emergent Behaviors

The development of AI in Age of Empires II without any prior knowledge of optimal strategies is an intriguing prospect. The AI has learned to farm, construct production buildings, and strategize without any human intervention. These emergent behaviors hold immense potential for creating innovative gameplay experiences.

Expanding AI Development

If you are interested in this project and would like to contribute or learn more, reach out to me in our Discord community. Although access to some files may be limited for security reasons, I am always willing to discuss the project, share insights, and engage in Meaningful conversations about genetic algorithms and AI development.

The Challenge of Implementing Resign Scripts

One of the upcoming challenges we face in AI development is implementing resign scripts. While we can hard-code a resign script to end the game, the effectiveness and consistency of this approach have been unreliable in past attempts. We will continue to explore solutions and Seek ways to enhance the overall AI experience.

Highlights:

• Shift from DE to AOC for faster training.
• Challenges in porting AI from User Patch 1.5 to DE.
• Promising behaviors in early AI training in AOC.
• Importance of villager production in the early game.
• Optimization of villager management for improved gameplay.
• Progression towards the Feudal Age and rewards system.
• Emergence of army training and strategic deployment.
• Inconsistencies in luring boars and opportunities for improvement.
• Fascination with the aesthetics of genetic algorithms.
• Potential of emergent behaviors in AI development.
• Expansion of the AI development community and future prospects.
• Challenges in implementing resign scripts for AI.

FAQ

Q: Is the AI compatible with the Definitive Edition of Age of Empires II? A: Currently, the AI is trained in the Age of Conquerors version of the game. Porting it to the Definitive Edition requires modifications that we are addressing.

Q: How does the AI prioritize its actions? A: The AI initially focuses on villager production and resource collection. As it learns and evolves, it will balance various strategies and adapt to different game scenarios.

Q: Can the AI conduct successful military raids? A: While the AI has shown the capacity to train armies, its ability to conduct raids and engage in battles is still a work in progress. However, we anticipate significant improvements in this area as the AI evolves.

Q: How can I contribute to the AI development project? A: Join our Discord community to connect with fellow enthusiasts and engage in discussions about the project. While access to certain files may be restricted, we welcome conversations and insights related to genetic algorithms and AI development.

Q: Are there plans to refine the AI's behavior regarding boar luring? A: Yes, we are actively working on addressing the inconsistencies in boar luring. Implementing consistent and effective boar luring tactics is a priority for further AI development.