Brain Implant Device Turns Signals into Speech using AI

07 May 2019


Scientists at the University of California, San Francisco have developed a brain implant capable of translating brain signals into synthesised speech.

The technology raises hopes of being able to one day restore speech to patients that have lost the ability to speak due to diseases such as ALS or Parkinson’s, stroke or brain injury. There is also a distant possibility of helping children with cerebral palsy learn to speak.

The team developed a two-stage method for translating thoughts into speech. The first stage involved placing electrodes on the brain of epilepsy patients that were already undergoing neural monitoring. The researchers then recorded the electrical signals that move the lips, tongue, voice box and jaw. They then determined how the movements in the mouth and throat would translate into audible sentences using AI and machine learning algorithms. The results then come out of a “virtual vocal tract” as synthesised speech.

The procedure could work for patients with many conditions but the technology relies on the parts of the brain which control the lips, tongue, voice box and jaw working correctly. So, patients with some types of stroke would not be able to benefit.

Professor Edward Chang, one of the researchers, said: “For the first time, this study demonstrates that we can generate entire spoken sentences based on an individual's brain activity.

“This is an exhilarating proof of principle that, with technology that is already within reach, we should be able to build a device that is clinically viable in patients with speech loss.”

The participants in the study were told not to make any specific mouth movements. Professor Chang said they were just asked to do the very simple thing of reading some sentences.

“So it's a very natural act that the brain translates into movements itself.” Professor Chang said, “We and others actually have tried to look at whether it's actually possible to decode just thoughts alone. And it turns out, it's a very, very difficult and challenging problem. That's only one reason why, of many, we really focus on what people are actually trying to say.”

In experiments, listeners accurately heard the correct sentence 43% of the time when given a set of 25 words to choose from, and 21% of the time when provided with 50 words. Although accuracy levels are relatively low, the technology will still help sufferers who are “locked in” feel more connected.