HeyGen neural network will dub your video in other languages – with voice and facial expressions
HeyGen has presented a technology that allows anyone to get a personal “diphone” – an artificial intelligence-generated duplicate of a video that can recite almost anything you type into a text field.
To get a personalized video avatar, customers have to send HeyGen a two-minute video of themselves speaking into the camera (from a smartphone) and then enter any text in a special field (a filter blocks explicit or violent content).
In addition to individual client avatars, HeyGen will offer a number of ready-to-use generic avatars and voices (they come in different genders and races).
This is what the result looks like:
The potential of the technology is both exciting and frightening (see below for an example of a diplomatic video with President Zelensky’s address in different languages).
However, HeyGen wants its tool to be used for “good” drip-feeds, such as marketing campaigns or creating training videos.
“We want to create a generative video engine that will replace cameras and allow everyone to create content freely,” said Joshua Xu, CEO of HeyGen, who worked at Snapchat for six years before launching his startup in late 2020.
Despite the fact that the tool has a lot of limitations (it doesn’t always pronounce proper names correctly, and lip movements don’t look perfect), the site already has huge queues and delays even when processing small videos.
HeyGen offers a number of subscription options ranging from about $50 to $150 per month with a certain number of credits, each good for one minute of video (higher-end packages offer the ability to create longer videos with higher resolutions).
The average cost is approximately $3 per minute, while setting up a personal avatar currently costs $199 (a simpler avatar based on a photo only is free; but there is also a professional avatar option for $1000 that requires the use of a studio and green screen). The free trial offers one credit per day, but only with standard HeyGen avatars or a talking photo.
