...
By Shane Croucher
Senior Editor
0
Share
Newsweek is a Trust Project member
See more of our trusted coverage when you search.Prefer Newsweek on Googleto see more of our trusted coverage when you search.
We're used to humans who sound like robots. The abject misery of dealing with one of the many bureaucracies of the state offers such opportunities in abundance. While this is dull and at times frustrating, it's mostly harmless.
But we are now in an age where robots can sound like humans: very convincingly, at that. Dessa, the new artificial intelligence model from Jack Dorsey's Block, is on the cutting edge of human mimicry. This is anything but dull—or harmless.
Dorsey said on X that it is "the most authentic full-duplex speech model ever built." It listens while it talks, cutting in, laughing mid-sentence, and murmuring along as people do.
Read More on Analysis
For decades, a robotic voice was a safety feature. It told you instantly that nobody was there. Dessa was built to remove that tell, for benevolent reasons. But with it goes one of the last easy ways to know who, or what, is on the line.
Lifelike AI voices aren’t new. What's new is how Dessa learned. Block says it was trained only on recorded human conversation, so it keeps every little sigh and giggle and stumble. It's a research preview with hand-picked demos, but the direction is clear.
Let's first consider the upsides. This will make it easier for the creative industries to use AI to generate more realistic productions.
It's a significant reduction in cost for them. That may open up the media to people bursting with ideas who would otherwise have struggled to gain access, or see their creative potential realized.
Beyond media, it could make automated help services feel more human and less, well, robotic. Elderly people calling health hotlines, for example, may find it easier to interact with an AI service that converses with them more naturally.
Then there's the epidemic of loneliness. A convincing AI may offer comfort to those who feel alone, sort of like a pet who can speak back to you beyond a woof or a meow.
But each upside has an evil twin. The savings that open up the media are lost jobs for actors and customer service agents.
Above all, the emergence of convincing AI voice models is an incredible gift to scammers. The warmth that reassures an elderly caller is what will disarm her when the voice belongs to a crook.
The art of a scam is in its ability to deceive the mark. That is becoming much easier.
Related Story
The Elizabeth Holmes Documentary Is a Masterclass in American Politics
Scammers already use AI to smooth the rough edges of their emails, the FBI warns. Instead of misspelled pleas from Nigerian princes, expect unexpected calls from a voice that sounds just like someone you know.
Until now, it is typically the more credulous or naive among us, including the clinically vulnerable, who fell for obvious scams. We are entering an age where many scams are no longer obvious at all, broadening the pool of who falls victim.
But the prince was never as dim as he looked. A 2012 Microsoft Research paper by Cormac Herley argued that the absurdity helps scammers. Replies cost them time, so an outlandish pitch screens out all but the most gullible.
AI that can run endless conversations removes that cost. Now everyone is a target.
Even scarier than having your bank account drained by a thief with an AI model is how this might impact politics and national security.
Political leaders have been duped by prank callers many times before, despite layers of supposed protection around them that limit access. What happens when the people leading our society are fooled by those with malevolent intentions?
We may soon find out. Last year, an impostor used an AI imitation of Marco Rubio's voice to contact three foreign ministers, a governor, and a member of Congress. That impostor left voicemails. The next one will be able to hold a conversation.
Related Story
The Future of American Power Runs Through India
Then there is the public itself. Two days before the 2024 New Hampshire primary, voters got robocalls from an AI imitation of Joe Biden that a consultant said cost him $150 to make. The truth is simply too slow.
Block's Dessa team acknowledges the risks and plainly wants to address them.
"A model that learned to talk from sound alone is new territory, and we want outside eyes on it," says Ragavan Thurairatnam, Dessa lead, in a message on its website titled "Calling all safety researchers!"
The team has been here before. In 2019, Dessa, then a Toronto startup, built a fake Joe Rogan voice but withheld the model, warning that such technology could help impersonate a politician to sway an election.
We are feeling our way through a daunting new world in which we can no longer believe that what we read, see, or hear is real. The technology giants are pressing ahead. Safety can't always keep up.
Ronald Reagan liked to quote an old Russian proverb to Mikhail Gorbachev: trust, but verify. It needs an update: never trust, always verify.
The FBI says to listen for tone and word choice to spot a cloned voice. Models like Dessa are built to pass that test. The bureau's other tip holds up better: agree on a family secret word.
View 0 Comments
Submit Correction
View Editorial & AI Guidelines