A
man paralyzed from the neck down due to a spinal cord injury he
sustained in 2007 has shown he can communicate his thoughts, thanks to a
brain implant system that translates his imagined handwriting into
actual text.
The device – part of a longstanding research collaboration called BrainGate – is a brain-computer interface (BCI), that uses artificial intelligence (AI) to interpret signals of neural activity generated during handwriting.
In
this case, the man – called T5 in the study, and who was 65 years of
age at the time of the research – wasn't doing any actual writing, as
his hand, along with all his limbs, had been paralyzed for several
years.
But during the experiment, reported in Nature earlier in the year, the man concentrated as if he were writing – effectively, thinking about making the letters with an imaginary pen and paper.
As
he did this, electrodes implanted in his motor cortex recorded signals
of his brain activity, which were then interpreted by algorithms running
on an external computer, decoding T5's imaginary pen trajectories,
which mentally traced the 26 letters of the alphabet and some basic
punctuation marks.
"This
new system uses both the rich neural activity recorded by intracortical
electrodes and the power of language models that, when applied to the
neurally decoded letters, can create rapid and accurate text," says first author of the study Frank Willett, a neural prosthetics researcher from Stanford University.
Similar systems developed as part of the BrainGate have been transcribing neural activity into text for
several years, but many previous interfaces have focused on different
cerebral metaphors for denoting which characters to write – such as
point-and-click typing with a computer cursor controlled by the mind.
It
wasn't known, however, how well the neural representations of
handwriting – a more rapid and dexterous motor skill – might be retained
in the brain, nor how well they might be leveraged to communicate with a
brain-computer interface, or BCI.
Here,
T5 showed just how much promise a virtual handwriting system could
offer for people who have lost virtually all independent physical
movement.
A diagram of how the system works. (F. Willett et al., Nature, 2021, Erika Woodrum)
In
tests, the man was able to achieve writing speeds of 90 characters per
minute (about 18 words per minute), with approximately 94 percent
accuracy (and up to 99 percent accuracy with autocorrect enabled).
Not
only is that rate significantly faster than previous BCI experiments
(using things like virtual keyboards), but it's almost on par with the
typing speed of smartphone users in the man's age group – which is about
115 characters or 23 words per minute, the researchers say.
"We've
learned that the brain retains its ability to prescribe fine movements a
full decade after the body has lost its ability to execute those
movements," Willett says.
"And
we've learned that complicated intended motions involving changing
speeds and curved trajectories, like handwriting, can be interpreted
more easily and more rapidly by the artificial-intelligence algorithms
we're using than can simpler intended motions like moving a cursor in a
straight path at a steady speed."
Basically,
the researchers say that alphabetical letters are very different from
one another in shape, so the AI can decode the user's intention more
rapidly as the characters are drawn, compared to other BCI systems that
don't make use of dozens of different inputs in the same way.
The man's imagined handwriting, as interpreted by the system. (Frank Willett)
Despite
the potential of this first-of-its-kind technology, the researchers
emphasize that the current system is only a proof of concept so far,
having only been shown to work with one participant, so it's definitely
not a complete, clinically viable product as yet.
The
next steps in the research could include training other people to use
the interface, expanding the character set to include more symbols (such
as capital letters), refining the sensitivity of the system, and adding
more sophisticated editing tools for the user.
There's
plenty of work to still be done, but we could be looking at an exciting
new development here, giving the ability to communicate back to people
who lost it.
"Our
results open a new approach for BCIs and demonstrate the feasibility of
accurately decoding rapid, dexterous movements years after paralysis," the researchers write.
"We believe that the future of intracortical BCIs is bright."
The findings are reported in Nature.