Snowflake Arctic takes pride as the "best LLM for enterprise AI." That's quite a big claim

Snowflake, a cloud-computing giant formed initially by former Oracle scientists, is now challenging big-time players in the AI war. The company launched Snowflake Arctic, its latest “best LLM for enterprise AI,” and claimed that it’s better than on par with Llama 3 70B & better than the latter’s 8B variant.

In its announcement, Snowflake claims that the Arctic model matches the performance of Llama 3 70B but with lower computing requirements and costs. It is touted as ideal for enterprise intelligence tasks in areas and benchmarks such as coding (HumanEval+ and MBPP+ ), SQL generation (Spider), and instruction following (IFEval).

That’s a big claim, especially considering that Llama 3 70B has been performing well against other major models like GPT-4 Turbo and Claude 3 Opus in important tests. Meta’s upcoming model reportedly scores well in benchmarks like MMLU (for understanding subjects), GPQA (biology, physics, and chemistry, and HumanEval (coding).

Snowflake Arctic mixes a 10B dense transformer with a 128×3.66B MoE MLP using a Dense-MoE Hybrid. This totals 480B parameters, but only 17B are actively used, chosen with top-2 gating. 

For small batch sizes like 1, Arctic reduces memory reads by up to 4x compared to Code-Llama 70B and up to 2.5x less than Mixtral 8x22B. But, as batch sizes increase significantly, Arctic becomes compute-bound. It incurs 4x less computing than CodeLlama 70B and Llama 3 70B.

You can try Snowflake Arctic on HuggingFace. The company also promises that the model will arrive soon on other model gardens like AWS, Microsoft Azure, Perplexity, and more.