Smart Speaker IQ test of Cortana, Alexa, Siri and Google Assistant shows most are smarter than ever

Loup Ventures have published the results of their semi-annual IQ test of digital assistants/ smart speakers, using a battery of 800 typical questions, and have found the speakers continue to get smarter and smarter.

The analysts chose to test smart speakers only, as they are less bound to the features of phones, giving the Amazon Echo (Alexa), Google Home (Google Assistant), HomePod (Siri), and Invoke (Cortana) a good going over.

The results hold few surprises, except for how much Siri has improved over the last year, making it now competitive with Amazons Alexa.

Over a 12-month period, Google Home improved by 7 percentage points, Echo by 9 points, Siri (9-month) by 22 points, and Cortana by 7 points in terms of questions answered correctly.

Of course while Cortana and Google Home both improved by the same amount, Cortana was from a very low base and is still trailing the others by a significant amount.

Some more of last year’s numbers can be seen below.

By category, the one surprise is how capable Google Home was in answering commerce-related questions e.g. where to buy an item, as Amazon’s Echo is somewhat hamstrung by its close ties to the Amazon Store.

Having no store and also no ties with retailers, Cortana was also the least helpful when it came to purchasing items.

While Google Assistant appears to be walking away with the crown, Loop Ventures is actually overall impressed with the whole category, noting:

With scores nearing 80-90%, it begs the question, will these assistants eventually be able to answer everything you ask? The answer is probably not, but continued improvement will come from allowing more and more functions to be controlled by your voice.

Ignoring Cortana, which is unlikely to show up in another smart speaker any time soon, it seems you can’t go much wrong by picking any ecosystem, rightly crowing 2018 as the year Smart Speakers went from an amusing toy to a commodity appliance.

Read Loop Venture’s full report here.

Thanks, John for the tip.