AI Reverse Engineers Super Mario Bros Just By Watching Gamers Play
Researchers are doing some wonderful things with machine learning and artificial intelligence technologies—gaining a better understanding of weather patterns, creating safer and more efficient transportation, predicting heart attacks, and even helping save a princess. Not just any princess, mind you, but Princess Peach. Researchers at the Georgia Institute of Technology's School of Interactive Computing have harnessed these awesome technologies to give AI a better understanding of how video games work, and they used Super Mario Bros. to do it.
Granting, teaching an AI to watch a video game and learn the mechanics sounds comparatively silly to some of the other things that are going on in the field, but that is missing the point. This is about building a smarter AI. The lessons that are learned can applied to other areas, just as IBM's Watson is capable of so much more than going on Jeopardy and beating the pants off of the greatest human contestants to ever appear on the show.
In this case, the researchers described in paper titled "Game Engine Learning From Video" how an AI can parse frames and essentially recreate the game engine of something like Super Mario Bros.
"At a high level, the game engine learning approach scans through the sequences of parsed frames and begins a search for a set of rules that explain any sufficient difference the predicted and actual frame," the paper states. ""If a game engine is found that reduces the difference to some threshold, then the scan begins again to ensure this new engine has not lost the ability to accurately predict prior frames."
The researchers note that this is a forward simulation model, albeit a unique one, via simple search over pixel input. They chose Super Mario Bros. for their initial test because it represents a physics system that is far less complex than reality, therefore easier to analyze and apply to AI.
While interesting, there are some limitations and drawbacks to what the researchers have done here. For one, the algorithm they used is comparatively slow. And secondly, the AI is not completely self-taught. The researchers fed it certain information, such as all the sprites in the game and concepts such as velocity. Still, it was up to the AI to use those tools and analyze each frame of Super Mario Bros., and then come up with rules that define the game play.
As for applying this research towards future technologies, the team hopes it will help with procedural and full-game generation. The researchers also believe this could scale up to more realistic domains outside of game play.
Image Source: Flickr (Amit Agarwal)
Granting, teaching an AI to watch a video game and learn the mechanics sounds comparatively silly to some of the other things that are going on in the field, but that is missing the point. This is about building a smarter AI. The lessons that are learned can applied to other areas, just as IBM's Watson is capable of so much more than going on Jeopardy and beating the pants off of the greatest human contestants to ever appear on the show.
In this case, the researchers described in paper titled "Game Engine Learning From Video" how an AI can parse frames and essentially recreate the game engine of something like Super Mario Bros.
"At a high level, the game engine learning approach scans through the sequences of parsed frames and begins a search for a set of rules that explain any sufficient difference the predicted and actual frame," the paper states. ""If a game engine is found that reduces the difference to some threshold, then the scan begins again to ensure this new engine has not lost the ability to accurately predict prior frames."
The researchers note that this is a forward simulation model, albeit a unique one, via simple search over pixel input. They chose Super Mario Bros. for their initial test because it represents a physics system that is far less complex than reality, therefore easier to analyze and apply to AI.
While interesting, there are some limitations and drawbacks to what the researchers have done here. For one, the algorithm they used is comparatively slow. And secondly, the AI is not completely self-taught. The researchers fed it certain information, such as all the sprites in the game and concepts such as velocity. Still, it was up to the AI to use those tools and analyze each frame of Super Mario Bros., and then come up with rules that define the game play.
As for applying this research towards future technologies, the team hopes it will help with procedural and full-game generation. The researchers also believe this could scale up to more realistic domains outside of game play.
Image Source: Flickr (Amit Agarwal)
