Google announces Gemini Ultra, its most capable AI model that can beat OpenAI GPT-4

Google today announced Gemini, its most capable and general model yet. As per the benchmarks shared by Google, Gemini model delivers state-of-the-art performance across many leading benchmarks. Yes, Gemini beats OpenAI’s GPT-4 in several benchmarks. Gemini 1.0 comes in different sizes: Gemini Ultra, Gemini Pro and Gemini Nano. These different versions are suitable for following different use cases:

• Gemini Ultra — our largest and most capable model for highly complex tasks.
• Gemini Pro — our best model for scaling across a wide range of tasks.
• Gemini Nano — our most efficient model for on-device tasks.

How Gemini compares with GPT-4:

• Gemini Ultra’s performance exceeds current state-of-the-art results on 30 of the 32 widely-used academic benchmarks used in large language model (LLM) research and development.
• With a score of 90.0%, Gemini Ultra is the first model to outperform human experts on MMLU (massive multitask language understanding), which uses a combination of 57 subjects such as math, physics, history, law, medicine and ethics for testing both world knowledge and problem-solving abilities.
• Gemini Ultra also achieves a state-of-the-art score of 59.4% on the new MMMU benchmark, which consists of multimodal tasks spanning different domains requiring deliberate reasoning.

Unlike early OpenAI GPT models, Gemini was developed natively to be multimodal, pre-trained from the start on different modalities. So, Gemini can easily understand and reason about all kinds of inputs.

Google Gemini roll out details:

• Bard will use a fine-tuned version of Gemini Pro for more advanced reasoning, planning, understanding and more. It will be available in English in more than 170 countries and territories, and we plan to expand to different modalities and support new languages and locations in the near future.
• Gemini is coming to Pixel 8 Pro.
• In the coming months, Gemini will be available in more of our products and services like Search, Ads, Chrome and Duet AI.
• Starting on December 13, developers and enterprise customers can access Gemini Pro via the Gemini API in Google AI Studio or Google Cloud Vertex AI.
• Android developers will also be able to build with Gemini Nano, our most efficient model for on-device tasks, via AICore, a new system capability available in Android 14, starting on Pixel 8 Pro devices.
• Gemini Ultra will be available to select customers, developers, partners and safety and responsibility experts for early experimentation and feedback before rolling it out to developers and enterprise customers early next year.
• Early next year, Google will launch Bard Advanced, a new, cutting-edge AI experience that gives you access to our best models and capabilities, starting with Gemini Ultra.

You can learn more about Gemini here.