Microsoft trained Phi-3 Mini only for a week with Nvidia's AI-friendly H100 GPUs

Microsoft launched the Phi-3 family of models, one of the best small models in the market at the moment. And now, Nvidia said and described how the Redmond company used its H100 GPUs to train these models, or more specifically, the Mini, 3.8B version.

“The model has 3.8 billion parameters and was trained on 3.3 trillion tokens in only seven days on 512 NVIDIA H100 Tensor Core GPUs,” says the tech maker on Tuesday. 

The family of Phi-3 comes with three variants: Phi-3 Mini (3.8B), Phi-3 Small (7B), and Phi-3 Medium (14B). It’s a massive improvement from the previous Phi-2 that was launched with just 2.7B parameters months ago. 

Phi-3 Mini, more specifically, also comes with two options depending on supporting tokens: 4K and 128K. You can try the latter at Nvidia’s AI center as an Nvidia NIM service for developers, and run the model locally using Windows DirectML or TensorRT-LLM.

“Phi-3 models significantly outperform language models of the same and larger sizes on key benchmarks (see benchmark numbers below, higher is better),” Microsoft said when launching the models, boasting that the Small and Medium versions can outperform larger models like GPT-3.5T. 

The Mini version is what’s available in the market at the moment, but Microsoft promised that all the other two models will be available shortly. You can also try Phi-3 Mini on Azure AI and Hugging Face.