Microsoft explains the importance of NPUs in Copilot+ PCs

Microsoft has made a lot of AI-centric announcements recently. Last week during the Build 2024 conference, the Redmond tech giant announced the future of computing, the Copilot+ PCs. It has dozens of features that apparently, no other PC could have it. One of them is the Recall feature, or “AI Explorer”, which lets you find anything on your PC and it goes beyond document searching. 

But how does it run? An NPU is an essential part of a Copilot+ PC, and it’s more cost-efficient and powerful than the CPU-GPU combo that most laptops released in the past years have. NPUs are specifically designed for neural networks and are very efficient. This leap in compute power allows Copilot PCs to run large language models locally on the device.

“The cool things that happened with our devices, especially Copilot+ PCs, is that we’ve kind of crossed that threshold. So now, our devices can run these types of (AI) models locally on the device to do amazing things,” Microsoft’s researcher Steven Bathiche explains in a video.

And that’s not baseless at all. An NPU chip inside a Copilot+ PC can run 45T operations per second with 4 watts less power. That powers 40 AI models for now.

One example of this new way of interacting with computers is the new Recall feature. As demonstrated in the video, Recall uses AI models to constantly monitor your activity in the background, such as OCR and image encoding, to help you remember what you’ve seen on your machine.

Although, it does raise some privacy concerns. Windows 11 can’t run this without scanning whatever you’re doing on your computer. Luckily, Microsoft cleared the air once and for all, saying that the Recall feature only works locally in the background. That means, Microsoft won’t use any data stored to train the model whatsoever.