VALL-E’s quickie voice deepfakes should worry you, if you weren’t worried already

But if you look back a bit, in as early as 2017 all you needed was a minute of voice to produce a fake version convincing enough that it would pass in casual use. And that was far from the only project.

Lyrebird is a voice mimic for the fake news era

The improvement we’ve seen in image-generating models like DALL-E 2 and Stable Diffusion, or in language ones like ChatGPT, has been a transformative, qualitative one: A year or two ago this level of detailed, convincing AI-generated content was impossible. The worry (and panic) around these models is understandable and justified.

Contrariwise, the improvement offered by VALL-E is quantitative not qualitative. Bad actors interested in proliferating fake voice content could have done so long ago, just at greater computational cost, not something that is particularly difficult to find these days. State-sponsored actors in particular would have plenty of resources at hand to do the kind of compute jobs necessary to, say, create a fake audio clip of the President saying something damaging on a hot mic.

I chatted with James Betker, an engineer who worked for a while on another text-to-speech system, called Tortoise-TTS.

Betker said that VALL-E is indeed iterative and like other popular models these days gets its strength from its size.

The emerging types of language models and why they matter

“It’s a large model, like ChatGPT or Stable Diffusion; it has some inherent understanding of how speech is formed by humans. You can then fine-tune Tortoise and other models on specific speakers, and it makes them really, really good. Not ‘kind of sounds like’; good,” he explained.

When you “fine-tune” Stable Diffusion on a particular artist’s work, you’re not retraining the whole enormous model (that takes a lot more power), but you can still vastly improve its capability of replicating that content.

But just because it’s familiar doesn’t mean it should be dismissed, Betker clarified.

“I’m glad it’s getting some traction because i really want people to be talking about this. I actually feel that speech is somewhat sacred, the way our culture thinks about it,” and he actually stopped working on his own model as a result of these concerns. A fake Dali created by DALL-E 2 doesn’t have the same visceral effect for people as hearing something in their own voice, that of a loved one or of someone admired.

VALL-E moves us one step closer to ubiquity, and although it is not the type of model you run on your phone or home computer, that isn’t too far off, Betker speculated. A few years, perhaps, to run something like it yourself; as an example, he sent this clip he’d generated on his own PC using Tortoise-TTS of Samuel L. Jackson, based on audiobook readings of his: