Microsoft patent propose using everyday objects as HoloLens controls

While Microsoft’s HoloLens technology is great for displaying information from the virtual world, actually controlling that presentation is still relatively awkward, involving a series of gaze and hand gestures which remain imprecise and hard to get right consistently.

Now in a new patent application Microsoft has proposed using gaze tracking and the everyday objects around you to help users access and control the virtual world.

Microsoft envisions HoloLens users being able to select objects in their enviroment by gaze, and then for these objects to be recognized and menus and actions being associated with it. In Microsoft’s example a worker looking at a picture of his wife on the desk would activate a display of the family calendar hovering above the picture,  which can then be accessed and manipulated by the user. That information may also be permanently attached to the picture, meaning if the picture is moved to another desk the information will still pop up next to it.

Microsoft also imagines movement of real life objects could be used to manipulate the virtual world. In their example looking at a basketball in the office would activate sports radio, and spinning the ball left or right in real life would raise or lower the audio playing in your ears.

The also imagine users being able to transfer documents to others by placing physical tokens on that person’s desk, with a digital document attached to it virtually, and the HoloLens adding a virtual glow to the object to get the users attention.

For this system to work users would need to wear their HoloLens near constantly, but we suppose there will come a time when the technology has shrunk down enough that this vision could become reality.