(Source: NEW SCIENTIST, Timothy Revell)
Artificial intelligence can put words right into your mouth. A new system takes a still image of a person and an audio clip, and uses them to create a doctored video of the person speaking the audio. The results are still a little rough around the edges, but the software could soon make realistically fake videos only a single click away.
It works by first identifying facial features using face-recognition algorithms. As the audio clip plays, the system then manipulates the mouth of the person in the still image so that it looks as if they are speaking. Very little pre-processing is required, so all of this can be done in real time.
“The application we’re thinking of is redubbing a video into another language,” says Joon Son Chung at University of Oxford, one of the creators of the system. In the future, the audio from news clips could be automatically translated into another language and the images updated to fit.
This isn’t the first system to automatically adjust images to new audio, but others have needed large amounts of video to work. They would pair up the way a person’s mouth moved when they made different sounds and then use that part of the image in edited footage.