NVIDIA Has GPT-4 Playing Minecraft And Its Bot Building Skills Are Impressive
As much as that sounds silly, it's true; AI can play Minecraft now, and is probably better at it than you are. Meet Voyager, what its developers call "the first LLM-powered embodied lifelong learning agent in Minecraft." That's a bunch of jargon, so let's break it down. "LLM" is "large language model," in this case GPT-4. These are AI models that have been trained on absolute reams and reams of text, and the basis of all convincing chatbots.
"Embodied" refers to the fact that the model itself controls a thing. This is usually a physical thing, like a robot arm, a drone, or an RC car, but in this case it's actually a virtual thing: Steve, the player character in Minecraft. Finally, a "lifelong learning agent" is an autonomous AI that can continue to learn new skills throughout its existence. Most "AI" is actually just a static black box that you put data into to be wrangled into a new form (e.g. separation of training and inference), but an AI with lifelong learning can continue to grow and adapt with its experiences.
So, Voyager is an autonomous AI entity that can play Minecraft, and it uses knowledge stored in GPT-4 to do it. Making a "bot" to play Minecraft is a historically-challenging task because Minecraft has no set structure or goals. The world is randomly-generated, events are unpredictable, and what you actually end up doing is ultimately up to the player. In that context, Voyager is extremely impressive.
In fact, Voyager is so far the best Minecraft-playing AI, apparently unlocking key tech tree milestones 15 times faster than the previous state-of-the-art. It accomplishes this through the use of three key modules: an automatic curriculum that maximizes exploration, a "skill library" of complex behaviors (like fighting or crafting), and an "iterative prompting mechanism" that generates code which is then fed to Minecraft to represent actions.
The automatic curriculum is generated by GPT-4, which was primed with the goal of "discovering as many diverse things as possible." This is essentially a novelty search, where the AI seeks out new things. GPT-4 takes as input a prompt that includes the objects around Voyager, the items in its inventory, its health and hunger, and other world state details from the game engine. Then, it generates tasks based on those parameters.
The developers say that Voyager demonstrates strong "in-context lifelong learning capabilities," which means that, without being trained on what constitutes "success" in Minecraft, it is able to learn the necessary skills to explore, survive, fight, and ultimately succeed in the goal of "discovering novel things." It's also able to do these things very rapidly compared to other models, as it is able to generalize already-learned skills to new contexts, like fighting spiders the same way it fought zombies. As long as that task is successful, it will be committed to the skill library.
Voyager isn't flawless; the authors admit that there are times when the agent gets stuck and fails to generate proper skills or recognize tasks as having been completed. GPT hallucinations also cause issues; the authors of the paper specifically mention the case where a task may be generated to craft an item that does not exist in the game, such as a "copper sword." The creators are confident that future improvements will minimize or remove these problems, though.
