Cortana delivers an embarrassing result in Digital Assistant IQ test

Microsoft’s home-grown digital assistant has delivered an embarrassing result in an independent test of its IQ.

When compared against other leading digital assistants in a battery of 800 questions by Loop Ventures they found Cortana dead last in its ability to answer queries.

In the test  Google Assistant was able to answer 86% of the questions correctly vs. Siri at 79%, Alexa at 61%, and Cortana at 52%.

In addition, Google Assistant was able to understand all 800 questions, while Siri misunderstood 11 questions, Alexa 13 questions and Microsoft’s Cortana 19 questions.

The questions were broken into 5 categories:

• Local – Such as “Where is the nearest coffee shop?”
• Commerce – Can you order me more paper towels?
• Navigation – How do I get to uptown on the bus?
• Information – Who do the Twins play tonight?
• Command – Remind me to call Steve at 2pm today.

It is only in navigation that Cortana did better than shelf-bound Alexa, but in every other category Cortana was last or tied for last with Alexa.

The digital assistant is also not keeping up with the improvement by the competition either.

Over a 15-month period, Google Assistant improved by 11 percentage points, Siri 13 points, and Cortana only 3 points in terms of questions answered correctly.

One proviso is that Cortana, Alexa and Siri were tested on an iPhone, which placed Alexa and Cortana at a disadvantage, but this is, of course, reflective of the real landscape of users and real-world usage, given that Microsoft currently lacks a mobile platform.

It is not clear if the result is due to a lack of effort or if Microsoft simply lacked the big data that Google, Apple and Amazon had access to by virtue of their large user base.  From earlier reports it appears Microsoft has already started throwing in the towel, refocussing Cortana from answering questions and command and control to assisting users in completing tasks, such as helping Windows 10 users to set up their PC. Microsoft has also invited Amazon’s Alexa onto PCs, which the results show should offer Windows 10 users a much better user experience than Cortana does at present.

Do the results show there is no place for a 4th Digital Assistant? Let us know below.

Source: Loop Ventures via WindowsLatest