Microsoft's Phi-2 2.7B model outperforms the recently announced Google Gemini Nano-2 3.2B model

Over the past few months, Microsoft Research has been releasing a suite of small language models (SLMs) called “Phi”. The Phi-1 was released first with 1.3 billion parameters and it was specialized for basic Python coding. In September, Microsoft Research released Phi-1.5 model with 1.3 billion parameters, but it was trained with a new data source that included various NLP synthetic texts. Despite its small size, phi-1.5 was delivering a nearly state-of-the-art performance when comparable to other similarly sized models.

Today, Microsoft announced the release of Phi-2 model with 2.7 billion parameters. Microsoft Research claims that this new SLM delivers state-of-the-art performance among base language models with less than 13 billion parameters. On some complex benchmarks, Phi-2 matches or outperforms models up to 25x larger.

Last week, Google announced Gemini suite of language models. The Gemini Nano is Google’s most efficient model built for on-device tasks and it can run directly on mobile silicon. Gemini Nano-like small language model enables features such as text summarization, contextual smart replies, and advanced proofreading and grammar correction.

According to Microsoft, the new Phi-2 model matches or outperforms the new Google Gemini Nano-2, despite being smaller in size. You can find the benchmarks comparison between Google Gemini Nano-2 and Phi-2 models below.

In addition to outperforming Gemini Nano-2, Phi-2 also surpasses the performance of Mistral and Llama-2 models at 7B and 13B parameters on various benchmarks. Find the details below.

While the previous two phi models were made available on Hugging Face, Phi–2 has been made available on the Azure model catalog. You can learn more about Phi-2 here.