Piano Genie AI Turns Anyone Into A Piano Virtuoso
Google does some wild things in its research that sometimes turn into cool products that we can fiddle with online. Now its Magenta research arm has created an AI that can turn anyone into a piano virtuoso. The creation is called Piano Genie and it's an AI program that allows anyone to improvise on the piano by pounding on eight buttons connected to a real player piano. Piano Genie was inspired by Guitar Hero, a game that was insanely popular a few years back and simplified playing the guitar, turning playing music into a game.
When team Magenta created Piano Genie, it took a different route than Guitar Hero, a game famous for allowing players to play pop and rock tunes written by well-known musicians. Piano Genie is meant to allow users to create a new melody on the fly using an AI program and a giant dataset of classical piano music. The AI can predict notes that should follow each other in a manner similar to how your smartphone can predict what word might follow next when creating a text message.
One of the creators of Piano Genie is Chris Donahue, an intern at Google Magenta, and one of the team of three working on the project. The other two members of the team include Ian Simon and Sander Dieleman; Dieleman is a veteran of Google's DeepMind. Donahue says that Piano Genie can improvise note to note to give the user a feeling of control over the music. He said when you are playing Piano Genie "occasionally, it will feel like it's sort of reading your mind and play the exact note you are intending to." He also notes that sometimes the AI will "disobey you" but that it still does something reasonable. This is why the system is called Genie says Donahue, because you can wish for what you want, but what you get isn’t always what you asked for.
The machine learning aspects of the project are constructed from common AI elements; the main component is a recurrent neural network that is good at learning to mimic sequential data like writing and music. That neural network was trained with a dataset of piano music taken from an international competition called the piano-e-competition. This was a particularly good dataset for the project because the competition records not only the notes played, but velocity as well, translating into timbre and volume. That data was used as the main training data for Piano Genie and built the predictive model of what piano notes follow each other.
The Piano Genie team then designed a pair of encoders that could combine that output into a format suited to their eight-button controller by compacting the notes from each of the 88 keys on a standard piano to work with eight buttons. The final portion was grafting what they had created to the self-playing piano seen in the videos. Piano Genie is available in a simulation online that you can try out; get your piano jam on here.
