YouTube has reportedly started rolling out a new AI-empowered translation feature for its content creators, one that will automatically redub a video’s contents into one of nine languages without changing the speaker’s voice.
According to a post from X user @levelsio, “YouTube will now auto dub videos in English, Spanish, Portuguese, German, French, Italian, Hindi, Indonesian and Japanese” and “will use AI to take the original voice but change the language.”
The new feature will arrive to creators’ channels in the coming weeks but will only be available to newly crafted content, rather than existing videos, to start.
I said podcasts but it's happening on YouTube first:
YouTube will now auto dub videos in English, Spanish, Portuguese, German, French, Italian, Hindi, Indonesian and Japanese
The dubs will use AI to take the original voice but change the language (like @lexfridman's podcast… https://t.co/1awdnDF8lk pic.twitter.com/KhelUxtLZL
— @levelsio (@levelsio) November 21, 2024
The auto-dubbing feature was first announced in September at the company’s “Made on YouTube” event. At the time, a YouTube spokesperson explained, “in the coming months we’ll be expanding our AI powered dubbing tool, formerly known as ‘Aloud,’ to hundreds of thousands of creators,” as well as adding additional languages beyond the English, Spanish, and Portuguese available for beta users. “Once creators have access, their videos will be automatically dubbed upon upload, with the ability to opt out if they so choose,” the spokesperson explained.
YouTube is not alone in its AI translation efforts. In 2023, Spotify announced a similar feature that reportedly creates “a more authentic listening experience that sounds more personal and natural than traditional dubbing.”
Upon its release, users were limited to translations between English and Spanish, though French and German have since been added. Meta has also recently released a “universal language translator” that it calls SeamlessM4T and is capable of swapping between any of more than 100 languages. Its abilities also extend to text-to-voice and text-to-text translations.
These new translation AIs could help content creators of all sizes reach new international audiences that they otherwise would be unable to. Of course, the technology could also backfire due to its tendency to hallucinate facts and responses. If it mistranslated your content into a language you don’t already speak, it’d be hard know about the mistake before viewer complaints started rolling in.