An algorithm to recover sounds from Silent Movies

Aug 19, 2014

Capgemini

Do you think we could recover intelligible speech, music, or ambient sound from the vibrations of a potato-chip bag photographed from 15 feet away through soundproof glass? Or extract useful audio signals from videos of aluminum foil, the surface of a glass of water, or even the leaves of a potted plant? Sounds like Science-Fiction, right?

Researchers at MIT, Microsoft, and Adobe have developed an algorithm that can reconstruct an audio signal by analyzing minute vibrations of objects depicted in video, and they will present their findings in a paper at this year’s Siggraph, the premier computer graphics conference (http://kesen.realtimerendering.com/sig2014.html).

Imagine you are in a closed room, with an empty potato-chip bag near you, and you are talking with a friend. The sound of your voice is going to make air all around you moving. And when sound hits an object, it causes the object to vibrate. These vibrations are invisible to the naked eyes, but with a high frequency camera, you can retrieve this information.

And no need of expensive cameras, researchers are already able to achieve the same results (with lower quality, but working fine) with regular 60fps video camera.

Now, “spies” could retrieve sounds from any room, just by using a video camera focused on a plant or a bag present in a remote room, without event filming people in the room, and even months later (if video is recorded).

Let’s have a look at this video below, and imagine all silent movies available in Internet, in National video archives, or even at your home you could use to recover ambient sounds and speech.

About the author

Sogeti Labs

SogetiLabs gathers distinguished technology leaders from around the Sogeti world. It is an initiative explaining not how IT works, but what IT means for business.

Comments