# Google Splits TPUv8 into Two Specialized Chips for Training and Inference
Google has announced a strategic split of the next-generation Tensor Processing Units TPUv8 into two independent lines: TPUv8t for model training and TPUv8i for inference. This approach reflects a shift in the AI industry—from general-purpose accelerators to specialized solutions optimized for specific types of computational workloads. Development is underway in collaboration with Broadcom (for training) and MediaTek (for inference), and the new chips will be tightly integrated with Axion processors based on the Arm architecture.
Specialization as a Response to the Growing Complexity of AI Workloads
Previously, TPUs served as versatile accelerators suitable for both training and inference. However, as models scale up and latency requirements tighten for processing user queries, this one-size-fits-all model becomes inefficient. As Jeff Dean, Google’s Chief Scientist, noted, “it makes sense to specialize in chips designed either for training or for inference.”
TPUv8t (codenamed Sunfish) is geared toward high-performance training of large language models. It’s optimized for large data batches, high memory bandwidth, and efficient gradient distribution across clusters. Meanwhile, TPUv8i (Zebrafish) focuses on minimizing inference latency, particularly in scenarios with dynamic requests from AI agents and chatbots.
This divergence enables:
- Reduced power consumption by ditching redundant features;
- Higher chip density in data center racks;
- Simplified cooling and power system design;
- Better compatibility with task-specific software stacks.
Integration with the Arm Ecosystem and Third-Party Software
The new TPUs will work in tandem with Axion processors—Google’s in-house development based on the Arm architecture. This signals a further shift away from x86 in infrastructure components. Integration at the system-on-chip (SoC) level and via interconnects ensures minimal latency between the CPU and accelerator.
What’s more, Google is testing TPU deployment in customer data centers, not just its own cloud infrastructure. This opens the door to third-party frameworks like PyTorch and orchestration tools beyond Google’s internal solutions. According to Amin Vahdat, Technical Director of Infrastructure, this move is essential for attracting enterprise customers who don’t want to be locked into one vendor’s ecosystem.
Scaling and Reliability Challenges
When operating hundreds of thousands of accelerators, even tiny manufacturing defects can lead to disastrous outcomes. As Paul Barham, one of the Gemini team leads, recounted, Google once faced a hardware glitch that caused the “complete self-destruction” of a model—training results turned invalid with no obvious failure signs. Diagnosing and fixing it took weeks.
Now the company is rolling out automated testing systems that can check hundreds of thousands of chips in 10 seconds. These systems simulate typical AI workloads and detect deviations in computation accuracy, even if they fall within the manufacturer’s specs.
Key challenges in scaling:
- Chip development cycle—around three years, while AI architectures evolve every 6–12 months.
- Software-hardware feedback loop—tight integration can hinder innovation, forcing developers to “think inside the box” of the current platform.
- Capacity shortages—limited production volumes create bottlenecks, especially with multi-billion-dollar deals from Anthropic and Meta.
Market Context and Competition
Nvidia remains the dominant force in training, but the inference market is fragmenting fast. Specialized solutions are emerging from Groq, Cerebras, SambaNova, and now Google. Meta’s interest in TPUv8i underscores how big companies are hunting for alternatives to cut latency and servicing costs for AI services.
Anthropic has already secured access to a million TPUs and plans to use them for advanced agent systems. G42 from Abu Dhabi and Citadel Securities are also actively testing the platform. Still, chip shortages are frustrating smaller customers hit with access limits.
What’s important:
- Google is splitting TPUv8 into two chips: TPUv8t (training) and TPUv8i (inference).
- Development with Broadcom and MediaTek, integrated with Arm-based Axion processors.
- New chips optimized for specific workloads, boosting data center efficiency.
- Google testing TPU deployment outside its clouds for PyTorch and other framework compatibility.
- Key risks: development cycles, capacity shortages, and software-hardware interdependencies.
— Editorial Team
No comments yet.