The first artificial intelligence algorithm to pass the “Turing Test for sound” has been created, capable of producing sounds that fool humans into thinking they are real.
Researchers from the Computer Science and Artificial Intelligence Laboratory (CSAIL) at Massachusetts Institute of Technology (MIT) demonstrated an algorithm that was able to view silent video clips and accurately predict realistic sounds that might appear.
In order to train the algorithm, the researchers fed it with around 1,000 videos containing an estimated 46,000 sounds produced using a drumstick.
“To then predict the sound of a new video, the algorithm looks at the sound properties of each frame of that video, and matches them to the most similar sounds in the database,” said Andrew Owens, a CSAIL PhD student who was the lead author on an upcoming study detailing the work.
“Once the system has those bits of audio, it stitches them together to create one coherent sound.”
Using this method, the algorithm was able to simulate the sound the drumstick would make when it was scraped, prodded or hit against an object.
Owens and his team believe the algorithm could be used to create accurate sound effects for movies, while the research could go on to help robots better understand the world around them.
“When you run your finger across a wine glass, the sound it makes reflects how much liquid is in it,” Owens said.
“An algorithm that simulates such sounds can reveal key information about objects’ shapes and material types, as well as the force and motion of their interactions with the world.”