For years, Pancho communicated by spelling out words on a computer using a pointer attached to a baseball cap, an arduous method that allowed him to type about five correct words per minute.
“I had to bend/lean my head forward, down, and poke a key letter one-by-one to write,” he emailed.
Last year, the researchers gave him another device involving a head-controlled mouse, but it is still not nearly as fast as the brain electrodes in the research sessions.
Through the electrodes, Pancho communicated 15 to 18 words per minute. That was the maximum rate the study allowed because the computer waited between prompts. Dr. Chang says faster decoding is possible, although it’s unclear if it will approach the pace of typical conversational speech: about 150 words per minute. Speed is a key reason the project focuses on speaking, tapping directly into the brain’s word production system rather than hand movements involved in typing or writing.
“It’s the most natural way for people to communicate,” he said.
Pancho’s buoyant personality has helped the researchers navigate challenges, but also occasionally makes speech recognition uneven.
“I sometimes can’t control my emotions and laugh a lot and don’t do too good with the experiment,” he emailed.
Dr. Chang recalled times when, after the algorithm successfully identified a sentence, “you could see him visibly shaking and it looked like he was kind of giggling.” When that happened or when, during the repetitive tasks, he’d yawn or get distracted, “it didn’t work very well because he wasn’t really focused on getting those words. So, we’ve got some things to work on because we obviously want it to work all the time.”
The algorithm sometimes confused words with similar phonetic sounds, identifying “going” as “bring,” “do” as “you,” and words beginning with “F” — “faith,” “family,” “feel” — as a V-word, “very.”
