Paralyzed man uses mind-powered robot arm to touch
Giving a high-five. Rubbing his girlfriend's hand. Such ordinary acts - but a milestone for a paralyzed man.
True, a robotic arm parked next to his wheelchair did the touching, painstakingly, palm to palm. But Tim Hemmes made that arm move just by thinking about it.
Emotions surged. For the first time in the seven years since a motorcycle accident left him a quadriplegic, Hemmes was reaching out to someone - even if it was only temporary, part of a monthlong science experiment at the University of Pittsburgh.
"It wasn't my arm but it was my brain, my thoughts. I was moving something," Hemmes says. "I don't have one single word to give you what I felt at that moment. That word doesn't exist."
The Pennsylvania man is among the pioneers in an ambitious quest for thought-controlled prosthetics to give the paralyzed more independence - the ability to feed themselves, turn a doorknob, hug a loved one.
The goal is a Star Trek-like melding of mind and machine, combining what's considered the most humanlike bionic arm to date - even the fingers bend like real ones - with tiny chips implanted in the brain. Those electrodes tap into electrical signals from brain cells that command movement. Bypassing a broken spinal cord, they relay those signals to the robotic third arm.
This research is years away from commercial use, but numerous teams are investigating different methods.
At Pittsburgh, monkeys learned to feed themselves marshmallows by thinking a robot arm into motion. At Duke University, monkeys used their thoughts to move virtual arms on a computer and got feedback that let them distinguish the texture of what they "touched."
Through a project known as BrainGate and other research, a few paralyzed people outfitted with brain electrodes have used their minds to work computers, even make simple movements with prosthetic arms.
But can these neuroprosthetics ever offer the complex, rapid movements that people would need for more practical, everyday use?
"We really are at a tipping point now with this technology," says Michael McLoughlin of the Johns Hopkins University Applied Physics Laboratory, which developed the humanlike arm in a $100 million project for DARPA, the Pentagon's research agency.
Pittsburgh is helping to lead a closely watched series of government-funded studies over the next two years to try to find out. A handful of quadriplegic volunteers will train their brains to operate the DARPA arm in increasingly sophisticated ways, even using sensors implanted in its fingertips to try to feel what they touch, while scientists explore which electrodes work best.
"Imagine all the joints that are in your hand. There's 20 motions around all those joints," says Pittsburgh neurobiologist Andrew Schwartz. "It's not just reaching out and crudely grasping something. We want them to be able to use the fingers we've worked so hard on."
The 30-year-old Hemmes' task was a much simpler first step. He was testing whether a new type of chip, which for safety reasons the Food and Drug Administration let stay on this initial volunteer's brain for just a month, could allow for three-dimensional arm movement.
He surprised researchers the day before the electrodes were removed. The robotic arm whirred as Hemmes' mind pushed it forward to hesitantly tap palms with a scientist. Then his girlfriend beckoned. The room abruptly hushed. Hemmes painstakingly raised the black metal hand again and slowly rubbed its palm against hers a few times.
These emotional robotic touches have inspired researchers now recruiting volunteers for soon-to-start yearlong experiments.
"It was awesome," is the decidedly unscientific description from the normally reserved Dr. Michael Boninger, rehabilitation chief at the University of Pittsburgh Medical Center. "To interact with a human that way. ... This is the beginning."
---
Hemmes' journey began in 2004. He owned an auto-detailing shop and rode his motorcycle in his spare time. Then one summer evening he swerved to miss a deer. His bike struck a guardrail. His neck snapped.
His determination didn't. Paralyzed below the shoulders, he's tried other experimental procedures in hopes, so far unrealized, of regaining some arm function.
"I always tell people your legs are great ... but they just get you from here to there," Hemmes says as his caregiver waits to feed him a bite of a cheeseburger near his home in Butler, north of Pittsburgh. "Your arms and fingers and hands do everything else. I have to get those back, I absolutely have to."
His ultimate goal is to hug his 8-year-old daughter. "I'm going to do whatever it takes, as long as it takes, to do that again."
Hemmes entered an operating room at UPMC with a mix of nerves and excitement.
"It's good anxiety," he says. "There is so much riding on this."
---
Think "I want that apple," and your arm reaches out and grasps it. You're not aware that neurons are instantaneously firing in patterns that send commands down the spinal cord - make the shoulder raise the arm, extend the elbow, flex the wrist and all five fingers.
A very similar firing occurs when you imagine movement or watch the movement you'd like to perform, explains Boninger, who with Schwartz is leading the Pittsburgh research together with a team of bioengineers, neuroscientists and physicians.
The DARPA arm was developed primarily for amputees. Separate research is under way to help them move it by using transplanted nerves to sense those brain commands. The paralyzed pose a more difficult challenge: getting those signals around a broken spinal cord.
For quadriplegic patients, scientists use implanted electrodes, called a "brain-computer interface" or BCI, to record that electrical activity. The signals move down through wires that tunnel under the skin and out by the collarbone, and are plugged into a computer or a robotic arm.
Until now, researchers mostly have tested miniature electrodes that poke inside the brain's motor cortex and record from individual cells, presumably allowing for precise movements. Pittsburgh's next test-patient will have two penetrating grids implanted in different parts of the cortex for a year to record from 200 cells altogether.
In contrast, Hemmes' chip sat on the surface of his motor cortex, a less invasive method that records from groups of cells. The size of two postage stamps, it's based on a kind of electrical signal mapping used to track seizures in epilepsy patients.
Both approaches need study, says Daofen Chen of the National Institutes of Health, who oversees neurorehabilitation research. He compares the options to eavesdropping on a party by sending in individual microphones or setting up a recorder at the window.
Boninger adds that scar tissue can blunt the penetrating electrodes over time, and the surface chips may be easier to convert to a wireless system, which is important for commercial use.
---
Hemmes' operation took two hours. He had practiced imagining arm movements inside brain scanners, to see where the electrical signals concentrated. That's where neurosurgeon Elizabeth Tyler-Kabara cut, attaching the chip through an inch-wide opening on the left side of Hemmes' skull.
Two days later, Hemmes was hooked to a computer, beginning simple cursor movements. The next week, it was time to test if he could trigger real-life movement using the DARPA arm.
Hemmes reclined in his wheelchair, the robot arm bolted to a steel rod nearby. The task: make the arm reach out to grasp a ball mounted on a board.
The arm whirs forward, then stops, then goes again, then suddenly pulls back.
"It's doing the opposite of what I ask it do," Hemmes says in frustration. "When I think about reaching back, it goes forward."
Dr. Wei Wang, a member of the research team, watches Hemmes' brain patterns on a nearby computer screen, trying to match them to the robotic movements. Focus on your elbow, Wang advises.
Hemmes takes a deep breath and tries. The arm whirs forward this time, reaching the ball. The fingers clench around it.
"There's no owner's manual," Hemmes says, thrilled that the back-and-forth pays off. "I'm training my brain to figure how to do all this."
Letting go is harder, the motor growling as the arm tugs backward before the fingers fully release. Hemmes starts imagining his hand relaxing before pulling backward, and the robot hand follows.
---
Sure, a robotic hand that one day mounts to a wheelchair could be useful. But no matter how well today's prosthetics move, they've got a problem: They don't sense what they touch. Normally, instant messages flash from the skin up to the brain to say "squeeze tighter" so we don't drop that coffee cup, or "tight enough" so we don't hug too hard.
Besides, Hemmes shares the dream of many quadriplegics. He doesn't want a bionic third hand. He wants to move his own hands again.
"These are all scientific goals that are very real," Boninger says.
Recreating sensation means crafting a two-way highway with those brain chips. That's what Duke University, in a study published last week in the journal Nature, did with its two monkeys. When the animals "touched" objects on a computer screen with their video game-like arms, electrical signals flashed back up to implanted electrodes - different signals for different textures, to tell the objects apart.
Sensors in the DARPA arm's fingertips allow for that same kind of feedback. McLoughlin says the plan is for one of the Pittsburgh study patients to begin testing touch capability next year, with a similar attempt at the California Institute of Technology to follow.
What about moving paralyzed limbs? Duke's plan is to turn its research into a robotic exoskeleton that would help the paralyzed move their bodies.
Hemmes is more intrigued by what's called functional electrical stimulation, zapping muscles with electrical currents to make them move. At Hemmes' request, Boninger's team attempted to fit his hand with a stimulator glove that might be linked to his electrode, but it was unsuccessful. The NIH's Chen says still other researchers are working on that kind of approach.
---
Hemmes likened moving the DARPA arm to learning to drive a car with a manual transmission. It took practice, but by week four he was moving the arm sideways as well as back and forth.
The fingers still clenched pretty tight, though. So when his girlfriend Katie Schaffer spoke up - "I want to hold your hand," she said on his last day of testing - Hemmes didn't dare bend them.
The two met after his accident, so he'd never before reached out to her.
"I was just trying to be gentle. I didn't want to hurt her, and I finally got there," Hemmes says. "Definitely the tears were flowing."
He says he was ready for a break after almost daily scientific testing, so removing the electrode and wires the next day wasn't a disappointment. He's confident the researchers will call him back once the technology advances.
