DeepMind's Crazy Good AI Is Now Demolishing Human Players In Quake III Arena
We often joke about the possibility of a Skynet situation playing out, but even if that never happens, there is at least one way that machines can assert their dominance over the flesh and blood race—gaming. We have already seen this play out in numerous ways, such as on Jeopardy when IBM's Watson wiped the floor with its opponents. Google's DeepMind has been very much active in this space as well, and is now adding Quake III Arena to the list of games where it can play as good as an actual person.
What's unique about Quake III Arena, as it relates to AI, is that the notion of capture the flag gameplay is a team-based sport that seemingly requires human traits to be effective at. Attacking and defending are critical elements, and it all hinges on effective teamwork to be successful. Apparently, though, computers can be good at this as well.
In a recent paper published in Science, researchers at an AI lab in London described the use of AI "agents" offering human-level performance in 3D games, using a technique called population-based reinforcement learning.
The researchers note that though AI agents have been getting better at two-player games, real-world problems that AI might be useful for (i.e., not just gaming) require teamwork. So, they designed a computer program that excels at one particular type, that being the capture the flag mode in Quake III Arena.
"The agents were trained by playing thousands of games, gradually learning successful strategies not unlike those favored by their human counterparts. Computer agents competed successfully against humans even when their reaction times were slowed to match those of humans," the researchers wrote.
The paper is a long read and goes into great detail on the technologies and training methods involved. Part of it entails repetition. By playing a game repeatedly, an AI agent can learn which strategies lead to success and which ones lend themselves to unfavorable outcomes, and adapt accordingly.
That's grossly oversimplifying things, of course, but the takeaway is that machines can learn to work together as a team, with the right training. That in and of itself leads to some interesting questions.
"How you define teamwork is not something I want to tackle," said Max Jaderberg, a DeepMind researcher who worked on the project. "But one agent will sit in the opponent’s base camp, waiting for the flag to appear, and that is only possible if it is relying on its teammates."
The ultimate goal is to use AI to tackle real-world problems, and not just win matches in Quake III Arena or decimate players in Dota 2. However, the lessons learned in one can be applied to the other.
