Microsoft announces Windows Semantic Index, Vector Embeddings and RAG API coming later this year

Microsoft yesterday announced a brand-new feature in Windows called Recall. Recall will help you find anything you’ve seen on your PC. You can just use natural language to search, and Recall will show you snapshots of the most relevant content.

Recall feature is grounded in several state-of-the-art AI models, including multi-modal SLMs, running concurrently and integrated into the OS itself. When you are using your PC, these models come across various kinds of content including text and images. Models transform these data and store it into a vector store. This native vector store available in Windows OS is called Windows Semantic Index and it is stored entirely on the PC.

To help developers bring similar language search capability in their apps, Microsoft will make Vector
Embeddings and RAG API will be available in Windows Copilot Library later. This will allow app developers to build their own semantic index store with their own app data. With the Retrieval Augmented Generation (RAG) API, developers can bring natural language search capability in their apps.