Project Tango is a Google project which is an attempt to create a mobile device unlike like any other, a mobile device that shares our sense of space and movement, that understands and perceives the world the same way we do. Google has developed a prototype device which is a 5” Android phone containing highly customized hardware and software designed to track the full 3-dimensional motion of the device as you hold it while simultaneously creating a map of the environment. These sensors allow the phone to make over a quarter million 3D measurements every second updating its position and orientation in real-time combining that data into a single 3D model of the space around you.
I recently came across a Microsoft patent which involves technology related to what Google is doing with Project Tango. Read the abstract of it below.
Mobile camera localization using depth maps is described for robotics, immersive gaming, augmented reality and other applications. In an embodiment a mobile depth camera is tracked in an environment at the same time as a 3D model of the environment is formed using the sensed depth data. In an embodiment, when camera tracking fails, this is detected and the camera is relocalized either by using previously gathered keyframes or in other ways. In an embodiment, loop closures are detected in which the mobile camera revisits a location, by comparing features of a current depth map with the 3D model in real time. In embodiments the detected loop closures are used to improve the consistency and accuracy of the 3D model of the environment.
Watch the video demo of Project Tango after the break.