The dangers of deepfake technology are obvious. They’re also scary. A.I. programs can make realistic renderings. For both faces and voices. Someone can make anyone they want, dead or alive, say anything they want. In ways that aren’t always easy to recognize. Even when that tech has full approval, it can still be ethically dubious. But like any technology, it’s not inherently good or bad. That depends on how it’s applied. There are wonderful reasons to use it. Like giving someone back his voice back after throat cancer took it away. Which is what one company did for Val Kilmer.
Val (currently available on Amazon Prime Video) features a lifetime’s worth of home movies from The Heat and Top Gun actor. However, Kilmer has difficulty speaking following a 2014 tracheotomy for throat cancer. So the documentary features Val Kilmer’s son Jack stepping in to record Val’s narration. But after the documentary was finished, Kilmer’s representatives reached out to Sonatic (which we learned about at Upworthy) to give Kilmer his voice back by digitally recreating it. The company uses a “powerful AI voice platform that enables entertainment professionals to create compelling, nuanced, and stunningly realistic voice performances.”
Typically Sonatic creates voice models by having actors record scripted performances. That audio is then fed into its proprietary Voice Engine. Which in turn trains the AI model. The replica voice can then say almost anything. But to capture Kilmer’s voice as it was, they had to work with what they had. Sonatic explained the process on its site:
“Our first hurdle with creating the voice model for Val was that his team provided a relatively small amount of recorded material, which we knew would make it more difficult to produce the model. We first cleaned the audio, carefully removing background noise without destroying the speech content. Next, we generated transcripts from the audio and paired the audio and text together in short chunks. The Voice Engine then began training the model with the resulting data—about 10 times less data than we might have used in a typical project.
Amazon Prime Video
The limited data resulted in subpar results. So the team researched and implemented new algorithms. They wanted to “produce higher-quality options from the data we had available.” Ultimately choosing “the best, highest-quality, most expressive one” out of 40 options.
Some use deepfake technology for nefarious reasons. But some use it for good. And giving someone back their voice will always be one of the best reasons to use it.
Originally published on August 19, 2021 and updated with the following correction: we originally stated the deepfake tech was used in the Val documentary and it was not.
