Microsoft takes the crown by topping 4 AI leaderboards

While Microsoft has not done anything as flashy as Google’s Deepmind beating Go, Microsoft’s AI researchers are still doing very well.

In a blog post, Steve Guggenheimer, Corporate Vice President of Developer Platform,  bragged about Microsoft’s achievements in the field, noting the company is the leader in four independent AI benchmarks with solutions which would be very applicable to addressing complex business intelligence challenges and delivering actional insights to enterprise customers.

The 4 leaderboards include:

1. SQuAD 1.1 (Stanford Question Answer Dataset):  The SQUAD tests asks an AI to read a document and answer questions related to this. In this machine reading comprehension (MRC) test Microsoft’s AI can now read and answer questions as well as a human.
2. Stanford SQuAD 2.0 Reading Comprehension test: In June 2018, SQuAD version 2.0 was released to “encourage the development of reading comprehension systems that know what they don’t know.” Microsoft currently occupies the #1 position on SQuAD 2.0 and three of the top five rankings overall on, while simultaneously maintaining the #1 position on SQuAD 1.1.
3. Salesforce WikiSQL Challenge: This uses a large crowd-sourced dataset based on Wikipedia, called WikiSQL, with the AI required to answer natural language questions from the dataset. Usually led by Salesforce, earlier this month Microsoft took the top position on Salesforce’s leaderboard with a new approach called IncSQL. The significant improvement (from 81.4% to 87.1%) in test execution is the result of collaboration between scientists in Microsoft Research and in the Business Application Group.
4. Allen Institute for AI’s Reasoning Challenge (ARC):  The ARC question answering challenge provides a dataset of 7,787 grade-school level, multiple-choice open-domain questions designed to test approaches in question answering. The top approach, essential term aware – retriever reader (ET-RR) was developed jointly by Microsoft Dynamics 365 + AI research team working with interns from the University of San Diego. The #3 position on the leaderboard is a separate research team comprised of Sun Yat-Sen University researchers and Microsoft Research Asia.

Microsoft notes that technologies developed by participating in these challenges are already finding their way into products which will be shipping as soon as October.

Read more about the tests and  Microsoft’s performance in Steve’s blog post here.