Samsung's Latest Deepfake AI Creates Talking Avatars From Still Photos
What if the person you were watching on TV was not an actual human? Researchers from the Samsung Institute in Moscow and Skolkovo Institute of Science and Technology have discovered a way to bypass 3D modeling. Samsung researchers recently introduced a new AI that can create talking heads from single photos and paintings.
It can be very difficult to create realistic talking heads without dipping into the uncanny valley. Companies such as Dessa have been able to replicate other’s voices, but videos are more difficult. Samsung researchers noted that “human heads have high photometric, geometric and kinematic complexity.” They also remarked that human beings are quite good at determining whether a talking head or robot is actually real. They argued that traditional methods of creating talking heads provide limited motion and head rotation. This lack of motion is easily detected by the human eye. Traditional methods also rely on huge amounts of images and these images can be difficult to access or take a long time to process.
Samsung’s new AI relies on three neural networks. The first is an embedded network that maps head images to vectors. The “generator” network then maps “input face landmarks” into output frames, or the videos that viewers see. The third network compares the resulting images and videos to the original source material and checks for fluency and accuracy.
Samsung’s AI consulted thousands of videos of celebrities to better understand important facial features and human movement. The combination of neural networks and learning from the YouTube videos now allows Samsung to create talking heads from single images. The more images that Samsung is able to access, the more realistic the talking head. Samsung provided examples of videos created with 1 image, 8 images, and 32 images. The videos that were developed with 32 images are more fluid and realistic. Samsung is even able to use historic photos and paintings to create talking heads. Their examples include talking heads of Albert Einstein and Leonardo da Vinci’s painting The Mona Lisa.
How does Samsung plan to use their AI? The researchers noted that the AI could be useful for “...videoconferencing and multi-player games, as well as special effects industry”. This kind of technology could also have immense effects on archaeology and history. Could you imagine being able to “watch” Abraham Lincoln give the Gettysburg Address? The AI could also unfortunately have negative effects on society. Ivan Mehta from The Next Web noted that AI has already been used to create harmful impersonations of celebrities. As with any technology, Samsung will need to be careful with how they use and market their latest AI achievement.
It can be very difficult to create realistic talking heads without dipping into the uncanny valley. Companies such as Dessa have been able to replicate other’s voices, but videos are more difficult. Samsung researchers noted that “human heads have high photometric, geometric and kinematic complexity.” They also remarked that human beings are quite good at determining whether a talking head or robot is actually real. They argued that traditional methods of creating talking heads provide limited motion and head rotation. This lack of motion is easily detected by the human eye. Traditional methods also rely on huge amounts of images and these images can be difficult to access or take a long time to process.
Samsung’s new AI relies on three neural networks. The first is an embedded network that maps head images to vectors. The “generator” network then maps “input face landmarks” into output frames, or the videos that viewers see. The third network compares the resulting images and videos to the original source material and checks for fluency and accuracy.
Samsung’s AI consulted thousands of videos of celebrities to better understand important facial features and human movement. The combination of neural networks and learning from the YouTube videos now allows Samsung to create talking heads from single images. The more images that Samsung is able to access, the more realistic the talking head. Samsung provided examples of videos created with 1 image, 8 images, and 32 images. The videos that were developed with 32 images are more fluid and realistic. Samsung is even able to use historic photos and paintings to create talking heads. Their examples include talking heads of Albert Einstein and Leonardo da Vinci’s painting The Mona Lisa.
How does Samsung plan to use their AI? The researchers noted that the AI could be useful for “...videoconferencing and multi-player games, as well as special effects industry”. This kind of technology could also have immense effects on archaeology and history. Could you imagine being able to “watch” Abraham Lincoln give the Gettysburg Address? The AI could also unfortunately have negative effects on society. Ivan Mehta from The Next Web noted that AI has already been used to create harmful impersonations of celebrities. As with any technology, Samsung will need to be careful with how they use and market their latest AI achievement.
