Meta next-gen MTIA chip for AI has finally arrived and it's faster

Meta has just recently unveiled its next-gen custom AI chips, boasting that they’re more powerful and well-focused on generative AI in mind: the next MTIA chip.

Meta Training and Inference Accelerator (MTIA) v1 first arrived last year in May. At that time, it was Meta’s first-generation AI inference accelerator tailored for its deep learning recommendation models. It also revamped various experiences across Meta’s properties like Facebook and Instagram. 

Now, Meta promises that the next version of MTIA will be faster than ever. In their own words, it “more than doubles the compute and memory bandwidth of our previous solution while maintaining our close tie-in to our workloads.”

In a revelation to TechCrunch, Meta details further what this chip is like. Even though it uses more power (90W compared to 25W), the next-gen MTIA has better features like more memory (128MB compared to 64MB) and runs faster (1.35GHz compared to 800MHz). It’s made using a smaller 5nm process, physically bigger, and has more processing cores.

Meta states that the new chips have been put into operation in their data centers, where they are actively serving models in production. They report positive outcomes from the program, emphasizing that the chips are capable of handling both simple and complex ranking and recommendation models. 

Meta asserts that their chips are more efficient than commercially available GPUs because they have control over the entire system.

A little while ago, Google also announced Google’s Axion AI chip, its debut foray into the market of dedicated chips for running models. Intel’s new Gaudi 3 AI chip also claims that it’s 50% faster than Nvidia H100.