Silent Speech Decoding Using Flexible Electronics and Artificial Intelligence

Abstract

An average adult speaks approximately 16,000 words per day, using verbal communication to build and maintain relationships, meet basic needs, navigate safely, and work. Approximately 10% of the US adult population reports a communication disorder, with severe disease preventing vocalized speech altogether. Although Augmentative and Alternative Communication technology is often used by people with communication disorders, existing systems are strongly limited in performance, inhibiting participation in spoken conversation, and creating an urgent need for improvement. Our approach uses grids of high-density surface electromyography (HD-sEMG) channels, embedded on a soft, conformable substrate, to enable close adhesion to the face during speech production and widespread coverage of articulator muscles. This enables us to infer wearer intentions with high accuracy. By combining novel materials science with modern machine learning, we aim to push HD-sEMG capabilities significantly beyond prior work and enable new forms of human-computer interaction.