To eliminate the discrepancies between the image and the dubbing, an American start-up has just developed a new tool based on deepfakes.
Among the cinephiles, there are two teams: those who prefer to watch a feature film in French so as not to have to read the subtitles and those who hate the discrepancies between the voice of the dubbing actor and that of the original actor. . A start-up could have developed the technology to reconcile the two camps. The company AI Flawless claims to have found the solution to this language barrier, which still prevents viewers from enjoying a foreign language feature film in the best possible way. According to them, the Deepfake could to synchronize the mouth of an actor, in several languages.
For those unfamiliar with the term deepfake, it is an artificial intelligence-based multimedia synthesis technique. It allows existing video files to be superimposed for example to replace the face of a character with that of another. It is thanks to this technology that Internet users have succeeded in creating a new version of Indiana Jones, with Chris Pratt as the famous adventurer.
Technology developed for more than three years by Flawless focuses on the mouths of the actors. She first uses the images in the original voice and adds the movements of the voice of the voice actor. Then, the two images are superimposed on each other to make it possible to create new lip movements in the actor. Then, Flawless offers a new perfectly (or almost) synchronized sequence. In a video, the company reveals the result is, it is quite impressive. So of course, we do not avoid a few bugs at certain times, especially when the character raises his voice, but it is already a revolution in itself. The founder of Flawless, Nick Lynes was also delighted with this feat. “When someone watches this dubbed footage, they’re not shaken by a jarring word or poorly timed mouth movement.” He nevertheless specifies that the experiment is not intended to completely transform the sequence. “It’s about keeping the original performance and style.”
Help the platforms in their conquest of the world
The goal of streaming platforms like Netflix or Disney+ is very clear: to impose itself on all markets, at all costs. But the language barrier is still a hindrance in this development. It must be said that not all spectators are ready to watch a work in its original version and that the absence of an effective dubbing can put off more than one. By using these tools, the giants of the industry could carve out a royal road in all the countries of the globe, by betting on foreign productions to seduce the most reluctant. As noted The Wallstreet Journal, as the American streaming market is saturated, the prospects for the development of platforms now lie beyond American borders. If this growth is accompanied by the creation of series and films in foreign countries, as with Lupin in France, Netflix’s large catalog also has strong arguments for conquering a new market. With the deepfake, for a much lower price than that necessary for a remake, streaming services like Netflix, Disney +, and even Amazon Prime Video, could in theory offer a drastically improved experience in all countries.