Google’s MuZero AI Gets All Queen's Gambit, Mastering Chess And Atari For Human Domination
Artificial intelligence is likely going to be a cornerstone of technology in the future. Until then, it needs to be improved, trained, and implemented well. Video games are an excellent way to train an AI due to the various paths and steps that can be taken, but it required some work to do. Now, the team at DeepMind has created an AI called MuZero, which can model and conquer games without even knowing the rules.
Back in April, DeepMind created an AI called Agent57, which was set out to tackle 57 Atari 2600 games from over forty years ago. The DeepMind team explains that the Atari games were chosen because “they provide a rich suite of tasks which players must develop sophisticated behavioural strategies to master, but they also provide an easy progress metric – game score – to optimise against.” Ultimately, the goal would be to train an AI to “think” about problems rather than just being trained to solve a specific one.
With Agent57, however, data and rules needed to be fed into the AI so it could learn and then estimate the best course of action. Due to the estimation rather than modeling all outcomes, Agent57 was also a model-free algorithm. Calculating all game outcomes can be rather computationally intense, so it was not really ever implemented. Now, MuZero simply learns the rules of the game and models only the most important information, which makes it incredibly efficient over typical model-based systems. The DeepMind team writes on its blog about the three key elements for MuZero’s models:
The value: how good is the current position?They liken these model elements to the weather, wherein rather than modeling all the raindrops, it would be more useful to know an umbrella will keep you dry.
The policy: which action is the best to take?
The reward: how good was the last action?
Overall, this AI seems to be an essential step toward AI that has better problem-solving skills. When problem-solving skills improve, AI can be applied to many more things than just games. It is a little bit like a child growing up, except it is an algorithm, learning and improving at an exceptionally fast rate. Perhaps we are on the way toward Skynet now.
