Microsoft Is Creating Immersive Telepresence, Realistic Physical “Body-double” Or Proxy In A Remote Meeting

Microsoft is working on some cool telepresence projects which was revealed by their latest job listings. Microsoft was always criticized for not utilizing their R&D well enough to make them into marketable products. Microsoft is trying to change this scenario with the new startup labs which will be productizing ideas and code from Microsoft Research.

Recently two new job listings revealed that a team is working on creating immersive telepresence products based on MSR inputs and current Microsoft communication products like Skype, Lync, etc,

The Microsoft Mesa project is creating immersive telepresence products that will change the way remote workers experience and are experienced in meetings. Mesa offers an equal blend of software and hardware challenges so you will get a chance to work on unique technologies that are unlike any other part of Microsoft. We build on top of proven technologies like Lync and Skype, but also use new emerging ones like Windows 8 and Surface, adding unique Mesa specific technologies (like video processing). Come learn new things in a small, fun team and get involved in every part of the product. Maximize your impact and live the startup life within Microsoft’ Startup Business Group.

Another job listing details that they are working on creating a realistic physical “body-double” or proxy in a remote meeting.

Want to be responsible for redefining how people telecommute, communicate and work? Not just getting them connected, but truly bringing them together with cutting edge software and hardware. How they look, how they express themselves, and how people connect will all be impacted by this groundbreaking new technology platform. Come be a leading developer on the team that will that will revolutionize communications and touch millions of customers around the world.

In the short term, we are developing the hardware and software necessary to have a realistic physical “body-double” or proxy in a remote meeting – one that gives the remote worker a true seat at the table, the ability to look around the room, turn to a colleague and have a side conversation. Longer-term, this same platform will enable high-definition communication scenarios for consumers over Skype. We are a small team, with a mix of short-term deliverables and long-term ambitions and we are developing quickly and having lots of fun in the process!

It looks like Microsoft is trying to integrate the ViewPort project from Microsoft Research. Here is the brief description of the project,

In this paper, we present Viewport, a fully distributed system for immersive teleconferencing. The participant at each site is captured by a camera rig, which contains multiple color and infrared cameras and multiple infrared projectors. The projected infrared dot pattern is used to help reconstruct a 3D representation in real-time, which will be rendered in a shared virtual environment for immersive telecommunication. We conduct a multimodal fusion scheme for multiview stereo and 3D reconstruction, which combines the knowledge of the infrared dot patterns from the projectors and the observations of IR and RGB cameras. One novel aspect of our approach is the use of sparse point cloud instead of dense multiview stereo for geometry reconstruction, which leads to significant speed up in 3D reconstruction and rendering. In addition, we introduce the scheme of virtual seating, where the point clouds are positioned to maintain the same seating geometry as face-to-face meetings after careful calibration, such that the mutual gaze between participants are faithfully maintained.

If you want to know how a startup environment works inside Microsoft, read the below description.

Responsibilities include implementing communications protocols, productizing ideas and code from Microsoft Research, designing APIs, and developing life-sized hardware for the conference room environment. Our team is small, so we don’t have specialized roles for testing or program management; everybody is expected to know the scenario, come up with the design, write code, test it, and deliver quality end-to-end. For this job, you need to be willing to get your hands dirty and own ideas from inception to delivery. In other words know the customer, know the use, and build it right. Then brag about it to your friends and family as they use it.

Check the job listing here and here.

via: @walkingcat