Disney Research Labs, an R&D arm of Disney that aims to “combine the best of academia and industry,” has just given an update on its progress with face-swap tech. In the video below, which comes via Engadget, the technology research group demonstrates how it’s been able to achieve “fully automatic” neural face swapping in images and videos. The researchers responsible for the breakthrough outline their novel face-swap process and summarize a paper written by Disney researchers outlining their new methods. Those methods were recently posted to Disney Research Studios.
According to the video, this high-resolution neural face swapping uses an unsupervised deep-learning method. That generates high-definition, photorealistic transfers of one face onto another. In other words: the face swaps achieved in the video are possible thanks to machine learning algorithms that are able to identify unlabeled patterns. The algorithms are able to “teach themselves” what makes up a “face” and what doesn’t. That way, they can subtract and apply it from and onto different people.
Disney Research Labs
The video outlines the facial transfer process beginning at about the 30-second mark. It shows how the researchers’ algorithms are able to detect a subject’s facial features, despite any movement of the face. The algorithms then take a pre-existing face, and map it onto those facial features. This blend results in the realistic, yet creepy, faces like the one immediately below (labeled “Ours”).
How exactly are these face-swaps are different from all of the other ones already out there? The researchers seem to be saying, in essence, that this method produces more realistic results. The video specifically notes that this face-swap method allows the generation of “high-definition video at a resolution and level of realism far beyond competing methods.”
Disney Research Labs
It’s easy to imagine how this kind of technology might be used in countless Disney movies and shows. And while the specifics of how it’ll be used aren’t clear, advancing the realism of CGI faces like those in The Irishman will probably be a high order. Maybe Disney will even want to bring cartoon faces into reality—for audiences who are into that kind of thing.
What do you think about Disney Research Labs’ new face-swap technology? Let us know in the comments!
Feature image: Disney Research Hub