Reverse image on Bing Visual Search may be getting smarter, latest Microsoft patent reveals

The reverse image is an important thing in this age of the internet. It lets you check out pictures and figure out if something’s legit or if someone’s pretending to be someone they’re not. Google Search has it, and so does Bing Visual Search from Microsoft. 

But it seems like Microsoft is set to take things further, as revealed in this recently published patent application. The USPTO has just launched the 37-page document for mass viewing in April 2024, although the patent itself was filed back in October 2023. 

This patent talks about creating personalized visual search systems. These systems customize search results based on your preferences. Then, they rank the results according to how likely you are to click on them.

In the presentation, Microsoft demonstrates two different users: one is a gardening enthusiast and the other is a chef. When they search for images of potatoes, the gardener gets gardening-related visuals of them, while the chef gets food images like French fries or mashed potatoes, for example.

This system finds potential matches using a technique called Approximated Nearest Neighbor (ANN) search. These matches are then sorted and ranked by an engagement-based ranker, which predicts how likely each image is to interest the user. 

Then, a diversity ranker further sorts the top results to make sure they’re visually diverse, and the system then presents these sorted images in no time.

Usually, when you upload a file for a reverse image search on the internet, the search engine gets back to you with images similar to the one you provide. But there’s a downside to this: it’s not personalized or tailored based on your needs. No matter who’s searching or why, they might show the same images for a given picture. This is where the patented system comes in as a solution.

Speaking of Bing Visual Search, you can also do a reverse image search on Copilot (formerly known as Bing Chat). This capability arrived some time ago, and with the personalized visual search systems being worked on, we expect this to also arrive in the AI assistance tool.