Tags
- PythonJavaScriptPyTorchReactAIGameFeatured
About
A Catan reinforcement learning agent created as part of a team project sponsored by CAE at the University of Florida's Integrated Product and Process Design (IPPD) program. IPPD is a capstone program that allows students to work on real-world projects with industry sponsors. For our team, Tactica, we were tasked with creating a machine learning agent that demonstrates tactical decision-making. We chose the game Catan due to its popularity, randomness, and strategic depth. We used an existing simulation, Catanatron, to simulate the game and provide a training environment for our agent. The simulation also provides a GUI, allowing users to play against the agent. We had to modify the simulation to add a trading interface, allowing the agent to trade with other players. I was responsible for most modifications to the GUI. We also added commands for training and testing, allowing for quick iteration and testing of our agents. For our agent, we used a reinforcement learning approach with a Graph Neural Network (GNN) to model the game board. The agent would choose among the many possible actions based on the current game state. When we found that this strategy was not effective, we pivoted to an ensemble approach where our agent instead picked among a set of strategies rather than a set of actions, reducing our action space. Our final agent was able to win at a rate of 98.4% against 3 random-choice players (players that choose random actions) and 22.7% against 3 of Catanatron's best players. Through this, we were able to demonstrate the potential of our unique ensemble approach. Our team wrote a paper detailing our methodology and results, which was presented at the Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC) in December 2025, where it received the award for Best Paper. This project would not have been possible without the help of my teammates: Max Banach, Andres Espinosa, Cody Flynn, Jason Li, Han Mach, and Cathy Quan. This project also would not have been possible without our coach, Dr. Jorg Peters, our CAE liaison engineer, Dr. Brian Stensrud, and the UF IPPD program led by Dr. Edward Latorre-Navarro.
External Links
Images
