For the first time, artificial intelligence has mastered teamwork in a complex first-person video game, coordinating its actions with both human and computer teammates to consistently beat opponents as a new study shows how researchers got AI bots to teach each other to work as a team.
The classroom was a simplified version of 1999 first-person shooter, Quake III Arena. The game involves two teams that navigate around a 3D map to retrieve a flag from their opponent’s base and return it to theirs. The team with the most captures after five minutes wins. Players also fire a laser to tag enemies, sending them back to their home base.
To train the AI to work as a team, the scientists created 30 different bots and pitted them against each other in a series of matches on randomly generated maps. The bots trained using brain-inspired algorithms called neural networks, which learn from data by altering the strength of connections between artificial neurons. The only data the bots had to learn from was the first-person visual perspective of their character and game points, awarded for things like picking up flags or tagging opponents.
Initially the bots acted randomly. But when their actions scored points, the connections that led to the behavior were strengthened through a process called reinforcement learning. The training program also culled the bots that tended to lose and replaced them with mutated copies of top performers inspired by the way genetic variation and natural selection help animals evolve.
After 450,000 games, the researchers arrived at the best bot, which they named “For The Win”. They then tested it in various matches with a mirror FTW, an FTW bot missing a crucial learning element, the game’s in-built bots, and humans. Teams of FTW bots consistently outperformed all other groups, though humans paired with FTW bots were able to beat them 5% of the time, they report today in Science.