Wheelchair users with severe disabilities can often navigate tight spaces better than most robotic systems can. A wave of new smart-wheelchair research, including findings presented in Anaheim, Calif., earlier this month, is now testing whether AI-powered systems can, or should, fully close this gap.
Christian Mandel—senior researcher at the German Research Center for Artificial Intelligence (DFKI) in Bremen, Germany—co-led a research team together with his colleague Serge Autexier that developed prototype sensor-equipped electric wheelchairs designed to navigate a roomful of potential obstacles. The researchers also tested a new safety system that integrated sensor data from the wheelchair and from sensors in the room, including from drone-based color and depth cameras.
Mandel says the team’s smart wheelchairs were both semiautonomous and autonomous.
“Semiautonomous is the shared control system where the person sitting in the wheelchair uses the joystick to drive,” Mandel says. “Fully autonomous is controlled by natural-language input. You say, ‘Please drive me to the coffee machine.’ ”
This is a close-up of the wheelchair’s joystick and camera.DFKI
The researchers conducted experiments (part of a larger project called the Reliable and Explainable Swarm Intelligence for People With Reduced Mobility, or REXASI-PRO) using two identical smart wheelchairs that each contained two lidars, a 3D camera, odometers, user interfaces, and an embedded computer.
In contrast to semiautonomous mode, where the participant controls the wheelchair with a joystick, in autonomous mode, control involves the open-source ROS2 Nav2 navigation system using natural-language input. The wheelchairs also used simultaneous localization and mapping (SLAM) maps and local obstacle-avoidance motion controllers.
One scenario that Mandel and his team tested involved the user pressing a key on the wheelchair’s human-machine interface, speaking a command, then confirming or rejecting the instruction via that same interface. Once the user confirmed the command, the mobility device guided the user along a path to the destination, while sensors attempted to detect obstacles in the way and adjust the mobility device accordingly to avoid them.
When Are Smart Wheelchairs Bad Value?
According to Pooja Viswanathan, CEO & founder of the Toronto-based Braze Mobility, research in the field of mobile assistive technology should also prioritize keeping these devices readily available to everyday consumers.
“Cost remains a major barrier,” she says. “Funding systems are often not designed to support advanced add-on intelligence unless there is very clear evidence of value and safety. Reliability is another barrier. A smart wheelchair has to work not just in ideal conditions, but in the messy, variable conditions of daily life. And there is also the human factors dimension. Users have different cognitive, motor, sensory, and environmental needs, so one solution rarely fits all.”
For its part, Braze makes blind-spot sensors for electric wheelchairs. The sensors detect obstacles in areas that can be difficult for a user to see. The sensors can also be added to any wheelchair to transform it into a smart wheelchair by providing multimodal alerts to the user. This approach attempts to support users rather than replace them.
According to Louise Devinge, a biomedical research engineer from IRISA (Research Institute of Computer Science and Random Systems) in Rennes, France, the increased complexity of smart wheelchairs demands more sensing. And that requires careful management of communication and synchronization within the wheelchair’s system. “The more sensing, computation, and autonomy you add,” she says, “the harder it becomes to ensure robust performance across the full range of real-world environments that wheelchair users encounter.”
In the near term, in other words, the field’s biggest challenge is not about replacing the wheelchair user with AI smarts but rather about designing better partnerships between the user and the technology.
This image shows data representations used by the 3D Driving Assistant. These include immutable sensor percepts such as laser scans and point clouds, as well as derived representations like the virtual laser scans and grid maps. Finally, the robot shape collection describes the wheelchair’s physical borders at different heights.DFKI
Where Will Smart Wheelchairs Go From Here?
Mandel says he expects to see smart wheelchairs ready for the mainstream marketplace within 10 years.
Viswanathan says the REXASI-PRO system, while out of reach of present-day smart wheelchair technologies, is important for the longer term. “It reflects the more ambitious end of the smart wheelchair spectrum,” she says. “Its strengths appear to lie in intelligent navigation, advanced sensing, and the broader effort to build a wheelchair that can interpret and respond to complex environments in a more autonomous way. From a research standpoint, that is exactly the kind of work that pushes the field forward. It also appears to take seriously the importance of trustworthy and explainable AI, which is essential in any mobility technology where safety, reliability, and user confidence are paramount.”
Mandel says he’s ultimately in pursuit of the inspiration that got him into this field years ago. As a young researcher, he says, he helped develop a smart wheelchair system controllable with a head joystick.
However, Mandel says he realized after many trials that the smart wheelchair system he was working on had a long way to go because, as he says, “at that point in time, I realized that even persons that had severe handicaps [traveling through] a narrow passage, they did very, very well.
“And then I realized, okay, there is this need for this technology, but never underestimate what [wheelchair users] can do without it.”
The DFKI researchers presented their work earlier this month at the CSUN Assistive Technology Conference in Anaheim, Calif.
This article was supported by the IEEE Foundation and a Jon C. Taenzer fellowship grant.
Instagram Engineering
Shopify Engineering