62 words per minute is about three times less than the speaking rate of an average conversation. But when it comes to a person whose facial paralysis prevents any oral expression, this score becomes prowess. A technical and scientific success that triples the maximum speed previously obtained, thanks to a neuroprosthesis placed against the patient’s brain and whose signals are then decoded by computer. “This team is recognized and their results are impressive“, comments the researcher at Inserm Jérémie Mattout, specialist in neuroscience, recalling however that caution is in order until they have been published and therefore validated by peers.
Decoding the brain to express yourself in writing
“We wanted to explore speech because of the potential for restoring rapid communication“, explains Frank Willett, neuroscientist at Stanford University (United States) and first author of these new work not yet published (pre-print). In 2021, his team had published in Nature the use of a similar method to allow a patient paralyzed to the neck to write “by hand”. In reality, a microchip with electrodes decoded the signals from the motor area of his brain while he imagined writing, so that the computer pulled out handwritten letters. The man was then able to express himself in writing, at a speed of 90 characters per minute and 99% accuracy. A score comparable to the 115 words per minute written on average by an able-bodied person on a smartphone.
“In this video, our subject tries to speak the sentences shown at the top of the screen, but his paralysis prevents him from producing intelligible speech. Our brain-machine interface decodes his neural activity and converts it into the desired message, writes at the bottom of the screen”, writes Frank Willett in the tweet below.
In this video, our participant tries to say the sentences shown at the top of the screen, but paralysis prevents her from producing understandable speech. Our brain-computer interface deciphers her neural activity and converts it into her intended message, shown at the bottom. pic.twitter.com/D2ep6mQjTn
— Frank Willett (@WillettNeuro) January 22, 2023
The first successful decoding of oral speech with a large vocabulary
Impressive results. But for Frank Willett and his colleagues, the future does not lie in written expression. “Normal speech is around 160 words per minute, which is much faster than handwriting. If successful, speaking can allow someone to participate in a conversation at a normal pace“, he explains. The idea had already been exploited in 2021 in the New England Journal of Medicine (NEJM) by a Californian team, but had only reached 15 words per minutewith 25% error.
To improve on this approach, this time the Stanford researchers used microelectrode arrays that record from single neurons instead of large groups of neurons. “The patient received four implants in the language motor control centers“, explains Jérémie Mattout. Each of the implants is a small matrix of 8×8 microelectrodes and therefore monitors about sixty neurons. Each movement of the patient at the level of the tongue, jaw, lips, etc., was classified only with the activation of the neurons , points out Jérémie Mattout, in order to predict or decode the phonemes that the patient was trying to pronounce.We believe this has allowed us to achieve higher performance by providing higher resolution brain signals.“, says Franck Willett. “We also use a more general decoding method that is not limited to a small set of 50 words“, as is the case in the publication of the NEJM. “We report successful decoding from a large vocabulary of 125,000 words“, he adds. “This is the first successful demonstration of decoding such a large vocabulary.” The error rate is still 24%, but was almost divided by three when the corpus was limited to 50 words.
62 words per minute, a speed that triples the previous record
The performance is impressive. “Our brain-machine interface (or ICM, editor’s note) decoded speech at the speed of 62 words per minute, which is 3.4 times faster than the previous record for any type of ICM and is beginning to approach natural conversation speed of 160 words per minute“, rejoice the authors in the publication. “There are several levels of innovation in this work. First, there are few human trials, then the results are impressively breaking records. We’re getting closer to classic communication speeds, it’s a leap forward“, supports Jérémie Mattout. A success which furthermore confirms that even after years of paralysis, the patient retains the possibility of activating the neural circuits allowing speech. “This is not always the case: sometimes patients with Locked-In Syndrome (total paralysis, editor’s note) are not able to use the interfaces, possibly due to associated cognitive impairment, at least this is a hypothesis we are currently exploring “, says the researcher.
However, it still remains to reduce the error rate by 24%, still too far from the 4 to 5% of the best “speech-to-text” systems (which can transcribe a written speech from a sound file) . “There is always a trade-off between speed and error rate. Here they have jumped into both!“, enthuses Jérémie Mattout. More electrodes, a finer decoding algorithm, many options remain to be explored in order to finally give fluent speech to those who are muscularly deprived of it while their brain has remained functional.”Let’s not forget that these systems are used in the laboratory, they are prototypes. Before imagining daily use, there are still many safety and practicality steps to validate.“