Microsoft patents Holodeck-style projection room (video)

The HoloLens field of view issue has been preoccupying Microsoft for some time, and they have been exploring a number of solutions, which tend to show up in their patent filings.

As Microsoft writes:

This discussion relates to complementary augmented reality. An augmented reality experience can include both real world and computer-generated content. For example, head-mounted displays (HMDs) (e.g., HMD devices), such as optically see-through (OST) augmented reality glasses (e.g., OST displays), are capable of overlaying computer-generated spatially-registered content onto a real world scene. However, current optical designs and weight considerations can limit a field of view (FOV) of HMD devices to around a 40 degree angle, for example. In contrast, an overall human vision FOV can be close to a 180 degree angle in the real world. In some cases, the relatively limited FOV of current HMD devices can detract from a user’s sense of immersion in the augmented reality experience. The user’s sense of immersion can contribute to how realistic the augmented reality experience seems to the user. In the disclosed implementations complementary augmented reality concepts can be implemented to improve a sense of immersion of a user in an augmented reality scenario. Increasing the user’s sense of immersion can improve the overall enjoyment and success of the augmented reality experience.

The latest patent, filed in June 2015 and published on the 22nd December 2016, is for COMPLEMENTARY AUGMENTED REALITY, postulates using a number of projectors to project images on the environment around you.

Crucially the images would be pre-distorted to complement your expected field of view from your HoloLens headset and would also take account of the depth map of the environment to intelligently change the images produced.

Microsoft notes:

In some implementations, multiple forms of complementary computer-generated content can be layered onto a real world scene. For example, the complementary content can include three-dimensional (3D) images (e.g., visualizations, projections). In another example, the complementary content can be spatially registered in the real world scene. Furthermore, in some implementations, the complementary content can be rendered from different perspectives. Different instances of the complementary computer-generated content can enhance each other. The complementary computer-generated content can extend a FOV, change the appearance of the real world scene (e.g., a room), mask objects in the real world scene, induce apparent motion, and/or display both public and private content, among other capabilities. As such, complementary augmented reality can enable new gaming, demonstration, instructional, and/or other viewing experiences.

The video below from Microsoft Research demonstrates the technique more convincingly.

We have of course seen multiple other solutions for the Field of View issue, and this one is probably the one that’s most extreme and demanding on external hardware, so it is unlikely we will ever see this come to real devices. We have however heard Microsoft is working on its own HoloDeck, so we may see this research pop up elsewhere eventually.

See the full patent here.