Meta Unveils Next-Gen Meta Training and Inference Accelerator (MTIA)
In a move to catch up with its rivals in the generative AI space, Meta has been investing heavily in its own AI efforts. While a significant portion of this investment is being allocated towards recruiting top-notch AI researchers, an even larger chunk is being devoted to developing specialized hardware for running and training Meta’s AI models. The latest manifestation of these efforts is the unveiling of the next-generation Meta Training and Inference Accelerator (MTIA), which boasts several upgrades over its predecessor.
A Closer Look at the Next-Gen MTIA
The successor to last year’s MTIA v1, the next-gen MTIA is built on a more advanced 5nm process, as opposed to the 7nm process used for its precursor. This reduction in process size allows for the creation of smaller and more efficient components, resulting in a physically larger design that packs more processing cores than the original. While this increased capacity does come at the cost of higher power consumption (90W compared to 25W), it also enables faster average clock speeds (1.35GHz up from 800MHz) and greater internal memory (128MB versus 64MB).
Improved Performance
According to Meta, the next-gen MTIA is currently operational in 16 of its data center regions and has delivered an impressive performance boost of up to 3 times compared to MTIA v1. While this figure may seem somewhat vague, Meta claims it was obtained by testing the performance of four key models across both chips.
Meta’s Hardware Showcase: A Sign of Slowing Progress?
Meta’s hardware showcase, which coincidentally took place just a day after Intel announced its latest AI accelerator hardware, is notable for several reasons. Firstly, despite being at the forefront of generative AI initiatives, Meta reveals in their blog post that they are not currently utilizing the next-gen MTIA for generative AI training workloads. Instead, the company has several ongoing programs exploring this possibility.
Secondly, Meta concedes that the next-gen MTIA will not replace GPUs for running or training models – a position that contrasts with the ambitions of its competitors. This suggests that while Meta is making strides in developing specialized hardware, it still lags behind its peers in achieving true independence from third-party GPUs.
A Reading Between the Lines
It’s clear that Meta’s AI teams are under pressure to cut costs. The company’s projected expenditure on GPUs for training and running generative AI models stands at an estimated $18 billion by the end of 2024, with training costs for cutting-edge generative models ranging in the tens of millions of dollars. In-house hardware presents a viable alternative, and it’s likely that Meta is eager to make progress in this area.
The Competition Heats Up
While Meta drags its feet, competitors are rapidly pulling ahead. This week saw Google unveil its fifth-generation custom chip for training AI models, the TPU v5p, as well as its first dedicated chip for running models, Axion. Amazon boasts several custom AI chip families under its belt, while Microsoft jumped into the fray last year with its own AI-focused hardware initiatives.
A Sign of Things to Come?
Meta’s next-gen MTIA may be a step in the right direction, but it remains to be seen whether this marks the beginning of a more significant shift towards self-sufficiency. With the competition intensifying and costs piling up, Meta will need to make rapid strides if it hopes to keep pace.
What This Means for the Future
The unveiling of the next-gen MTIA highlights the ongoing struggle between Meta and its competitors in the generative AI space. While progress is being made, there’s still a long way to go before true parity is achieved. As the stakes grow higher and costs continue to rise, one thing is clear: only time will tell if Meta can bridge this gap.
What You Need to Know
- Next-gen MTIA features: 5nm process, physically larger design with more processing cores, increased internal memory (128MB), faster average clock speeds (1.35GHz)
- Performance boost: up to 3 times compared to MTIA v1
- Current deployment: operational in 16 of Meta’s data center regions
- Generative AI use case: several ongoing programs exploring this possibility, but not currently utilizing next-gen MTIA for training workloads
Stay Ahead of the Curve
To stay up-to-date on the latest developments in generative AI and specialized hardware, be sure to subscribe to TechCrunch’s newsletters. By staying informed about the rapidly evolving landscape of AI research and innovation, you’ll be better equipped to navigate the complex challenges and opportunities that lie ahead.
