New Microsoft patent would make Holograms less ghostly

One of the most prominent visible features of the Microsoft HoloLens is the light shade in front of the lenses, designed to reduce the intensity of outside light and thereby make the holograms generated by the light field lenses more visible.

Microsoft notes however that as the HoloLens finds more application outdoors even this light shade cannot be enough to prevent holograms from being washed out by ambient light and take on a ghostly appearance.

They write:

Near-eye display (NED) devices used for example in augmented reality are ideal for use both indoors and outdoors. However, ambient illumination levels span a wide range, from 10 lux in dim interiors to 120,000 lux in outdoor direct sunlight. NED devices must insure AR hologram visibility for all ambient brightness levels. However, as NED devices are used in increasingly brighter environments, e.g. outdoors, display power consumption can become problematic. Even at high power levels, AR holograms may appear washed out in comparison to real world objects in the bright ambient environment. Additionally, especially in bright environments, real world objects and backgrounds may be visible through the AR hologram, giving them a ghostly, less realistic appearance. Moreover, some external light sources passing through NED viewing optics can be optically transformed into visible artifacts that may appear as brightly colored rainbows and/or colored ghosts.

Microsoft, however, has a solution that does not involve cranking up the brightness of the holograms but instead is all about smartly and selectively dimming the outside world in the areas that are superimposed by the hologram, thereby preventing the holograms from being overpowered by the outside world.

Microsoft writes:

Embodiments of the present technology relate to a system and method for controlling an amount of ambient light transmitted to the eye of a wearer through an NED device. Embodiments of the present technology may include both passive and active components for controlling transmitted light. A passive component may comprise a photochromic coating applied to a visor of the NED device. The photochromic coating may be responsive to UV light, for example from sunlight, to darken and limit the amount of light which passes through the visor.

An active component may comprise a bi-layer monochromatic dimming panel assembly positioned behind hologram forming optical assemblies. The dimming panel assembly can reduce ambient light viewed through the NED device indoors or outdoors. A pixel level opacity mask may be developed each frame which may be used by the dimming panel assembly to form dark masks behind augmented reality (AR) holograms and semi-dark masks that create the appearance of drop shadows. The positions of bright light sources in a scene can also be identified, and those positions defined in the pixel level opacity mask so that the dimming panel assembly can block light from those bright light sources.

The technology would in short use a cheap monochromatic LCD panel which can selectively dim at different levels, and use a system to identify which particular pixels coincide with the generated hologram, and dim those, in particular, making them more vibrant and real looking.  The panel could even be used to generate realistic drop shadows.

The advantage of the technology is of course that it is relatively simple and cheap. Microsoft also suggests the use of a photochromic layer on the larger light shield which would respond interactively to the intensity of the outside light, becoming much darker when outside and more transparent when inside, when such a high level of protection is not needed. I can imagine the same technology could also be used to turn an Augmented Reality headset to a VR headset relatively easily, especially if the field of view of the AR screen is improved.

The patent dates to December 2016. Microsoft is said to be working on anew, improved and cheaper HoloLens 2 for early 2019. Hopefully, we will see some of these smart innovations included in the final product if and when it finally arrives.

The full patent can be seen here.