2023-09-14 21:03:24
A woman on the computer screen asks, with slightly strained green eyes, “How does my voice sound?” The audio keeps cutting out in an image that clearly appears to have been created on a computer, but it is nonetheless a very surprising moment. This image is a digital avatar of a woman who suffered from aphasia as a result of a stroke that occurred 18 years ago. She took part in an experiment using her brain implant and an AI algorithm that allowed her to speak with an artificial voice that replicated her own voice. In addition, she was able to convey her facial expressions, albeit in a limited way, through the avatar.
Recently, two research teams <네이처(Nature)>to publishedone thesisThey showed that the field of brain implants is developing very rapidly. However, these studies are still at the concept verification stage, so there is a long way to go before the general public can use this technology. Each study was conducted on two female participants who had lost the ability to speak. One of them is suffering from the followingeffects of a brainstem stroke, and the other is unable to speak due to the followingeffects of Lou Gehrig’s disease.
In the study, each participant had a different recording device implanted in their brain. Then they were able to speak 60 to 70 words per minute. This is regarding half the normal speaking speed, but more than four times faster than previously reported. Researchers led by neurosurgeon Edward Chang of the University of California, San Francisco, also captured brain signals that control the subtle movements that make up facial expressions. This allowed the researchers to create avatars that expressed the participants’ words in almost real time.
“This paper demonstrates a very sophisticated and rigorous approach to brain science and engineering,” said Judy Illes, a neuroethicist at the University of British Columbia in Vancouver, Canada, who was not involved in either study. Illes especially appreciated the addition of expressive avatars. She said, “Communication is not just words exchanged between people, but words and messages conveyed through intonation, facial expression, stress, and context.” “I think it was a thoughtful attempt,” he added.
Chang’s research team has been studying this problem for over 10 years. In 2021, they demonstrated that they might capture brain signals from patients who had suffered brainstem strokes and transcribe them into words or sentences, albeit slowly. The latest paper used a larger implant with the number of electrodes increased to two. The implant, a device regarding the size of a credit card, was used to capture brain signals from a patient named Ann, who was unable to speak following suffering a stroke regarding 20 years ago.
The implant does not translate thoughts, but rather captures the electrical signals that control the muscle movements of the lips, tongue, jaw and larynx that form speech. “For example, to make a ‘P’ or ‘B’ sound, you have to purse your lips,” said Alexander Silva, an author of the paper and a graduate student in Chang’s lab. “So electrical signals from the brain will activate specific electrodes that control the lips.” The researchers transmitted this signal to a computer through a terminal attached to the participant’s scalp and decoded it with an AI algorithm. Additionally, to increase accuracy, an automatic correction function using a language model was used. Using this technique, the researchers translated Ann’s brain signals at 78 words per minute using a vocabulary of 1,024 words. At this time, the error rate was 23%.
Going further, the researchers succeeded in interpreting brain signals directly into speech for the first time. And the muscle signals they captured allowed participants to express three different emotions – happiness, sadness and surprise – at three levels of intensity through their avatars. “Speech conveys not only the words but also who the speaker is,” Chang explained. “Voice and facial expressions are also part of our identity.” Anne, a participant in this research trial, told researchers, “My dream is to become a counselor.” She thinks clients may feel more comfortable with this avatar. The researchers used voice recordings from her wedding video to make the avatar sound like her voice.
Another research team, the Stanford researchers, announced their research results in a preprint paper in January 2023. Researchers implanted four tiny implants the size of aspirin pills that might record signals from single neurons in a patient named Pat Bennett with Lou Gehrig’s disease. Bennett trained the system by reading syllables, words, and sentences out loud 25 times.
The researchers then tested her skills by having her read sentences that were not used during system training. When she tested sentences drawn from 50 vocabularies, the error rate was regarding 9%. When the vocabulary was expanded to 125,000 to include most English words, the error rate increased to regarding 24%.
Artificial voices through brain-computer interfaces are not smooth. Slower than normal speech. The error rate is around 23-24%, which is better than before, but is still not enough. Sometimes the sentence is reproduced perfectly, but sometimes the question ‘How is your cold?’ is incorrectly expressed in a similar way (‘Your old’).
But scientists are confident that this technology can be developed further. “Interestingly, the more electrodes you add, the better the interpretation performance,” said neuroscientist Francis Willett, author of the Stanford research paper. “If you increase the number of electrodes and neurons, you can get even more accurate.”
In reality, the systems used in these studies are not suitable for use at home. Participants were unable to leave the laboratory and communicate with the outside world through their brain implants, as they had to connect wires to their heads and analyze the signals on a large computer. “There is still a lot of work to be done to translate this knowledge into something useful for people,” said Nick Ramsey, a neuroscientist at the UMC Utrecht Brain Center in Amsterdam and a commentator on the previous two papers. .
Illes also warns that each research team reports results from a single participant, and that the results may not apply equally to people with similar brain disorders. “This study is at a proof-of-concept stage,” she said. “We know that brain damage is very complex and manifests itself in many different ways, and whether this can be generalized even within the stroke or Lou Gehrig’s disease population is possible, but we cannot be certain. “It’s a problem that doesn’t exist,” he said.
Nonetheless, this technology shows the potential for new technological solutions for people who have lost their ability to communicate. “We have proven that this technology has the potential to be a solution and that there is a way to achieve it,” Chang said.
Ignition ability is very important. Participants in the window study usually communicated using tablets. “I was so sick of my husband getting up every time to translate my tablets,” she said. “We didn’t even argue because he didn’t give me a chance to rebut. “Of course I was dissatisfied!” she said.
1694740734
#era #opens #stroke #patients #aphasia #communicate #digital #avatars #MIT #Technology #Review