Microsoft’s Kinect peripheral senses motion inputs using a combination of a depth sensor, RGB camera and a multi-array microphone. Now Microsoft’s research division, Microsoft Research, has developed a gesture recognition system called SoundWave that relies on sound instead of video to bring gesture recognition capabilities to a standard laptop or desktop computer without the need for any additional hardware.
The system uses a PC's existing speakers to emit an inaudible tone between 18 and 22 kHz. As the sound waves reflect off a moving object they change in frequency. The PC’s microphone detects this frequency change - known as the Doppler Effect - along with the sound wave’s speed, amplitude, direction and time variation.
Based on this information, an algorithm then translates the movements into gestures, which can be used to control things onscreen. SoundWave can detect multiple objects - or hands - and even works in noisy environments or when the laptop is being used to play music.
While the Kinect has found great success as a gaming peripheral with its ability to work at a bit of a distance, SoundWave appears to only detect gestures made relatively close to the computer. This might limit the technology to deskbound applications, but the number of unforeseen applications to which the Kinect has been adapted suggests SoundWave could find equally surprising uses for those chained to a desk.
The research team has already developed some examples that demonstrate the potential for the technology. Along with the standard scrolling with a wave of the hand, the responsiveness of the system has been demonstrated by using it to play Tetris. It could also be used in an office environment to automatically lock a user’s screen when they leave their desk.
The SoundWave technology is still under development in the Microsoft Research labs, and it’s not clear if or when we can expect to see this kind of gesture-recognition system released. While it looks promising, Microsoft might want to maximize the return on its Kinect for Windows before releasing gesture-recognition technology that does away with the need for purchasing any additional hardware.
