Neuroengineers have made a revolutionary device that uses neural networks for machine learning to read brain activity and translate it into speech.
In an article in Scientific Reports, Tuesday, it is described in detail that the team at the Zuckerman Brain Institute at Columbus University has deeper learning algorithms and the same kind of technology that drives devices like Apple's Syria and Amazon Echo to transform thoughts and understandable reconstructed speech . "but a magazine article enters much greater depth.
The human-computer framework could eventually provide patients who have lost the ability to speak the opportunity to use their thoughts to communicate verbally through synthesized robotic voice.
"We have shown that, with the right technology, the minds of these people can be decrypted and understandable to every listener," says Nima Mesgarani, lead researcher of the project.
When we talk, our brain shines, sending electrical signals around the old thought box. If scientists can decode those signals and understand how they are related to word formation or listening, then we are approaching speech translation. With enough understanding – and enough processor power – that could create a device that directly translates speech into speech.
And that's what the team did, creating a "vocoder" that uses algorithms and neural networks to convert speech signals.
To do this, the research team asked five epileptic patients who had already undergone brain surgery. They connected the electrodes to different exposed areas of the brain, then the patients listened to a pronounced sentence of 40 seconds, which they accidentally repeated six times. Listening to the story helped train the vocoder.
Then the patients listened to the speakers ranging from zero to nine while their brain signals went back to the vocoder. Vocoder algorithm, known as the WORLD, then splashed its own sounds, which were cleansed by the neural network, which eventually resulted in robotic speech that mimics counting. You can hear how it sounds here. It's not perfect, but it's certainly understandable.
"We've found that people can understand and repeat the sounds about 75 percent of the time, which is far above and above all the previous attempts," Mesgarani said.
The researchers concluded that the accuracy of the reconstruction relies on how much electrode was placed on the patient's brain and how long the vocoder was trained. As expected, increasing electrodes and increasing training length allows the vocoder to gather more data and result in better reconstruction.
Looking ahead, the team wants to test what signals are emitted when a person just imagines speech, as opposed to listening to speech. They also hope to test a more complex set of words and sentences. Improving more data algorithms could ultimately lead to brain implantation bypassing speech by turning thoughts into words.
That would be a big step forward for many.
"Those who lost their ability to speak, either through injury or illness, have renewed the opportunity to connect with the world around them," Mesgarani said.
NASA has spent 60 years: The Space Agency has taken mankind away from anyone and has plans to go further.
Taking Extreme: Combine crazy situations – volcano eruption, nuclear meltdown, 30-wave waves – with day-to-day technology. Here's what's going on.