Patent suggests Microsoft worked on gesture and eye-tracking accessory for the Surface Pro

Microsoft’s recent addition of eye tracking to Windows 10 Fall Creators Update Insider builds has been well received, but the technology does require specific hardware which does not work well physically with laptops and tablets, due to the requirement for the hardware to be mounted on the front of the computer.

It seems, however, Microsoft has been considering a way around this issue that would allow them to bring support for depth, gesture and gaze sensing technology to their Surface Pro tablet.

Microsoft proposes a module which would make use of the Surface keyboard connectors’ data and power ports while being held securely in place by the connectors and their magnets.

The module, in turn, would have slots for the Surface Keyboard, acting as a pass through for that accessory and therefore not interfering with the normal function of the device.

The cameras would be angled such that they would be pointed at the area over the keyboard, to capture natural hand gestures, but may also include a field of view sufficient for gaze tracking and face recognition.

The technology for gaze and depth sensing is of course mostly mature, making the main innovation the unique attachment and modular nature of the technology.

One of the inventors is Kfir Karmon, Project Prague Lead & Engineering Manager at Microsoft Advanced Technology Labs. Project Prague allows app developers to include basic hand gesture support into their applications without much work. They can also create customised gestures for their own apps or other products, with very little additional programming or expertise.   A video of the technology can be seen here.

The other inventor is Adi Diamant, Director of Advanced Technology Labs Israel, with specialities in innovation and New User Interfaces amongst others.

It is not known if Microsoft is still pursuing the idea, and we have certainly not seen the many prototype Surface keyboards (such as the DJ one) Microsoft demoed come to the market, but with a new emphasis on accessibility and natural user interface, we hope Microsoft does not leave this innovation on the drawing board.

The full patent can be seen here.