Silent Speech Recognition

  • A participant using the silent speech interface
  • Design of the silent speech interface
  • Researchers working on movement-driven synthesizer to help patients who have trouble with speech

Project Overview

People with laryngectomy (surgical removal of the larynx due to cancer) and speech motor impairment (e.g., due to neurological diseases) struggle to communicate with others because of their impaired oral communication. Limited treatment options are available to improve the quality of their speech.

The purpose of the Silent Speech Recognition project is to develop an articulatory movement-driven speech synthesizer that can enable those patients to speak using their tongue and lips (rather than by typing on an AAC device, which is limited by the slow manual input). The current focus of this project is to develop algorithms that can convert articulatory movement time-series data to text with high speed and accuracy. In the future, the algorithms will be integrated with a portable data collection device and text-to-speech synthesizer.Small Group Intervention of Language Expression

Stage of Development

This project is currently in the early stage of development. Thirteen technical papers have been published, based on data collected from healthy speakers. At the UT Dallas Communication Technology Center, data collection from speakers with impaired motor speech for the next stage of development has recently started, funded by the National Institutes of Health (NIH) through a R03 grant (PI: Wang).

Development Team

The Silent Speech Recognition project is a collaboration among the Callier Center for Communication Disorders and Department of Computer Science, UT Dallas; Department of Computer Science and Engineering, University of Nebraska-Lincoln; and Massachusetts General Hospital in Boston. The team of researchers includes:

Jun Wang, PhD [email protected]
The University of Texas at Dallas, Biomedical Engineering and Communication Sciences and Disorders

Jordan R. Green, PhD [email protected]
Institute of Health Professions, Massachusetts General Hospital (MGH), Department of Communication Sciences and Disorders

Ashok Samal, PhD [email protected]
University of Nebraska-Lincoln, Department of Computer Science and Engineering

Balakrishnan Prabhakaran, PhD – [email protected]
The University of Texas at Dallas, Erik Jonsson School of Engineering and Computer Science