Microsoft's AllHands can analyze big chunks of feedback easier with language processing

Good news for software developers. Microsoft’s researchers have recently announced AllHands, an analytic framework that uses natural language processing and large language models (LLMs) to analyze large volumes of verbatim feedback.

Collaborating with ZJU-UIUC and the National University of Singapore, AllHands starts by organizing feedback into categories and topics, then uses LLMs to make the process more accurate and user-friendly. 

More specifically, AllHands includes an LLM agent that allows users to ask questions using everyday language. This agent then translates these questions into Python code, which runs on the feedback. This enables AllHands to respond to various formats, including text, code, tables, and images.

The framework surpassed existing methods in classification, topic modeling, and query response across diverse datasets. Microsoft and co. tested AllHands using three different sets of feedback. At every step, from sorting and categorizing the feedback to giving accurate answers to questions, AllHands did better than the others.

Microsoft says that verbatim feedback, which includes users’ experiences and opinions, is crucial for improving software, but getting useful information from this feedback can be hard and time-consuming. This is where AllHands comes in as a solution. 

“The analysis of such feedback data plays a pivotal role in streamlining the product release and upgrade processes for developers. This entails addressing existing issues, introducing new features, and resolving user confusion, ultimately contributing to the overall enhancement of the product,” the paper reads.