Microsoft's SIGMA is made for real-world tasks. This is how AI enters your eyes through HoloLens
Me for real-world tasks for HoloLens 2
2 min. read
Published on
Share this article
Improve this guide
Read our disclosure page to find out how can you help MSPoweruser sustain the editorial team Read more
Key notes
- Microsoft introduces SIGMA, an open-source platform for real-world tasks
- SIGMA blends mixed reality and AI, offering step-by-step guidance through HoloLens 2
- Tasks can be pre-defined or dynamic, with language and vision models aiding object detection and user queries
Microsoft researchers have just announced on Monday that it launched SIGMA. Short for Situated Interactive Guidance, Monitoring, and Assistance, it’s an “open-source research platform and testbed prototype” made for real-world tasks.
Now available on GitHub, SIGMA is an interactive tool designed for HoloLens 2 that blends mixed reality and AI tech. It assists users in performing step-by-step tasks by providing guidance and answering questions, or in simpler words, it lets AI enter your eyes.
Tasks can then be pre-defined or generated dynamically, and SIGMA uses language and vision models to detect and highlighting relevant objects in the environment. It can also answer user questions and detect task-relevant objects in the environment using vision models like Detic and SEEM.
“SIGMA was designed to serve as a research platform. Our goal in open-sourcing the system is to help other researchers leapfrog the basic engineering challenges of putting together a full-stack interactive application,” Microsoft says.
Built on the Platform for Situated Intelligence framework, SIGMA enables fast prototyping and provides tools for visualization, debugging, and maintenance. The system is designed with a client-server architecture, allowing data streams from HoloLens 2 to be processed on a desktop server, bypassing device limitations.
Microsoft has been at the forefront of the AI race for quite some time. The Redmond-based company’s billions of dollars in investment in OpenAI, its numbers of AI models, and the Copilot AI tool in Windows 11/10 and mobile devices are just to name a few.
