Instant NeRF can transform 2D images into a 3D scene in a few milliseconds

It really is the era of modern technology where things can be done in just a matter of seconds. That is proven by Instant NeRF, which was able to render a 3D scene featuring a model dressed like Andy Warhol in a demo conducted by the NVIDIA Research team. It only took the process tens of milliseconds to do it using still photos taken at different angles.

According to NVIDIA, the concept of Instant NeRF is pretty simple: using inverse rendering and applying it to neural radiance fields, or NeRF, one can transform a set of 2D images into a 3D work in just a snap. This is far from the traditional methods of rendering a 3D scene, which usually takes hours to days depending on the details you want to include. Early NeRF models that use AI somehow shorten the rendering period, though not that significant. Then here comes the Instant NeRF, the method that can do it in milliseconds. 

“If traditional 3D representations like polygonal meshes are akin to vector images, NeRFs are like bitmap images: they densely capture the way light radiates from an object or within a scene,” says David Luebke, NVIDIA Graphics Research Vice President. “In that sense, Instant NeRF could be as important to 3D as digital cameras and JPEG compression have been to 2D photography — vastly increasing the speed, ease and reach of 3D capture and sharing.”

The Instant NeRF includes a fast 2D image capture process of capturing the still shots, as motions from the subjects can translate to a blurry 3D scene. After this, the NeRF will recreate the scene by supplying the missing information and predicting the color of light radiating in any direction in the 3D space.

“It relies on a technique developed by NVIDIA called multi-resolution hash grid encoding, which is optimized to run efficiently on NVIDIA GPUs,” writes Isha Salian, part of NVIDIA’s corporate communications team, in a blog post. “Using a new input encoding method, researchers can achieve high-quality results using a tiny neural network that runs rapidly… The model was developed using the NVIDIA CUDA Toolkit and the Tiny CUDA Neural Networks library. Since it’s a lightweight neural network, it can be trained and run on a single NVIDIA GPU — running fastest on cards with NVIDIA Tensor Cores.”

According to NVIDIA, the Instant NeRF can be useful for a wide variety of purposes, including creating avatars and virtual worlds or reconstructing scenes and events in 3D form. It can also be used in helping and training the robots to learn the size, shape, and actual dimensions of objects.