For better or worse, it’s getting easier than ever to doctor video footage, and the latest development in this field is as scary as it is impressive. A new algorithm developed by researchers from Stanford University, Max Planck Institute for Informatics, Princeton University, and Adobe makes it possible to alter human speech in a video, just by changing the text in its transcript.
This method is said to alter the video while preserving the speaker’s characteristics. To accomplish this, the algorithm first reads the phonemes and pronunciation of letters and words from the original video, and creates a model of the speaker’s head to accurately replicate the speaker’s voice and movements.
Once you edit the transcript, the algorithm performs a search for segments containing lip movements that make the words you’ve typed in, and replaces the original phrase. But the replaced part can have a lot of pauses and cuts, because it’s stitched together tiny segments of video from across the clip. So, the algorithm applies some intelligent smoothing to make the edited video appear more natural.
We humans aren’t smart enough to not trust what our eyes and ears tell us. If we see a video of someone saying something, we’re going to believe she said it.