A team from the University of Washington (UW) has developed ground-breaking software called Vocal Joystick, which will enable people with disabilities to control their computers using the sound of their voice without the use of a mouse. This virtual computer mouse which is driven by sound has been tested at the UW Medical Center with spinal-cord-injury patients. The researchers hope they will have a prototype available this fall.

This software doesn’t use standard voice-recognition technology. Instead, it detects basic sounds at approximately 100 times per second and utilizes them to generate fluid cursor movement. Researchers maintain that this system is easier to use because it allows users to use a large set of sounds for both continuous and discrete movement…as well as the ability to make visual adjustments on the fly.

The VJ system consists of three main components:

• acoustic signal processing

• pattern recognition

• motion control

The processing module then extracts short-term acoustic features. Signal conditioning and sensitive analysis techniques are necessary for accurate analysis of these features. The next step is to place these  features into the pattern recognition module where energy smoothing, pitch and formant tracking, vowel classification and discrete sound recognition occur.  The last step is for energy, pitch, vowel quality and discrete sound to be transformed into direction, speed, and other motion related parameters.

The application driver takes these parameters and then launches corresponding actions.