Research shows how HoloLens can act as eyes for blind users (video)

Researchers from Caltech and University of Southern California have published a new research paper discussing how Microsoft HoloLens can act as eyes for blind people.

To restore vision of blind people, some approaches have been explored to convey raw images to the brain. These approaches were not effective because of the bandwidth requirements and the extensive training required to interpret unusual stimuli. In this new research paper, researchers have proposed an alternate approach. Instead of trying to convey the whole sensory data to the brain, their new approach restores vision at the cognitive level.

They are using Microsoft HoloLens for this purpose. As you all know, HoloLens can recognize the environment around its user, knows the user’s position in the environment and it can also capture video and other data for detailed scene knowledge. Researches use these data and convey them to blind people through auditory augmented reality.

This system supports many aspects of visual cognition: from obstacle avoidance to formation and recall of spatial memories, to long-range navigation. Neither training nor modification of the physical environment are required: Blind subjects can navigate an unfamiliar multi-story building on their first attempt. The combination of unprecedented computing power in wearable devices with augmented reality technology promises a new era of non-invasive prostheses that are limited only by software.

There researchers also tried their system with actual blind people. They found that blind users were able to reliably locate and point to objects from audio cues. For example, they were able to to find a chair in a room in a fraction of the time they normally would. You can read the full research paper here and see the technology in action in video below.

Via: Techcrunch