Point, Shoot, Connect: Microsoft's New Vision for Accessory Pairing

Microsoft is developing a new system that lets you connect Surface Pen (which works with the new GPD Duo laptops by the way), or even any kind of electronic accessories, a lot easier by simply turning on your camera and facing them.

The patent application, which was filed back in December 2022 but has just recently been published in June 2024, details how this system could work. That’s honestly a lot more convenient than the traditional method of connecting to Bluetooth and holding a button on the accessories at the same time.

Basically, say you have a tablet with a front camera. This camera is then able to take a picture of your stylus pen and uses a machine-learning model to analyze that image to find and pair the accessory automatically. This camera can work continuously or be triggered by certain signals, and control in the device handles the image analysis for accurate detection.

In fact, it goes even deeper than that. Microsoft further says that the model—thanks to suitable ML and AI techniques—can distinguish between a stylus held in a hand and one that is resting or charging so that it knows which one is ready to connect.

It prioritizes accessories close to the camera or in specific positions, like earbuds near ears, and estimates distances using image size or depth cameras.

If the device is not already paired with the detected accessory, it checks for other accessories using methods like RF signals or audio output signals. But then, even if your accessory isn’t nearby or in the camera’s view, it will warn you about a potential pairing mistake.

“Machines may be implemented using any suitable combination of state-of-the-art and/or future machine learning (ML), artificial intelligence (AI), and/or natural language processing (NLP) techniques,” Microsoft then says.

Thoughts on this new, potential feature?