Alex Kipman talks about the new best-in-class depth sensor in the next generation HoloLens

At Build Day 1 developer conference, Microsoft showed off a new Project Kinect for Azure, a package of sensors, including Microsoft’s next generation depth camera, with onboard compute designed for AI on the Edge.

This new sensor package will include Microsoft’s breakthrough Time of Flight sensor with additional sensors all in a small, power-efficient form factor. This sensor package will make use of Azure AI to significantly improve insights and operations. This can enable fully articulated hand tracking and high fidelity spatial mapping, enabling a new level of precision solutions. On LinkedIn, Alex Kipman confirmed this 4th generation Kinect sensor will find its way into the next generation of the Microsoft HoloLens.

The sensor features:

• Highest number of pixels (megapixel resolution 1024×1024)
• Highest Figure of Merit (highest modulation frequency and modulation contrast resulting in low power consumption with overall system power of 225-950mw)
• Automatic per pixel gain selection enabling large dynamic range allowing near and far objects to be captured cleanly
• Global shutter allowing for improved performance in sunlight
• Multiphase depth calculation method enables robust accuracy even in the presence of chip, laser and power supply variation.
• Low peak current operation even at high frequency lowers the cost of modules

The current version of HoloLens uses the third generation of Kinect depth-sensing technology to enable it to place holograms in the real world. The next generation HoloLens will use Project Kinect for Azure, to integrates with Microsoft’s intelligent cloud and intelligent edge platform.

Microsoft has already said they will also use a next-generation Holographic Processing Unit which includes AI capabilities, enabling on-device deep learning.  Doing deep learning on depth images can lead to dramatically smaller networks needed for the same quality outcome. This results in much cheaper-to-deploy AI algorithms and a more intelligent edge.

Microsoft has still not revealed when the next generation of HoloLens will be coming, but with the device having passed its second birthday it is believed to be sooner rather than later.