Google Cloud: Trillium TPUs reach general availability

2024-12-12 · Publisher : Google Cloud · Domain : Cloud & DevOps

Google Cloud opens Trillium, its sixth-generation TPU used to train Gemini 2.0, to all customers. Google claims more than 4x the training performance and 67% better energy efficiency than the previous generation.

### Background On 12 December 2024, Google Cloud announces general availability of Trillium, its sixth-generation TPU (Tensor Processing Unit), announced earlier in the year. Google says it used Trillium to train Gemini 2.0, its most advanced model at the time. Enterprises and startups thus gain access to the same infrastructure as Google. ### Key facts - Compared with the previous generation, Google claims: more than 4x training performance, up to 3x inference throughput, 67% better energy efficiency, 4.7x peak compute per chip, and doubled HBM memory capacity and interconnect bandwidth. - Up to 100,000 Trillium chips in a single Jupiter network fabric, with 13 petabits per second of bisectional bandwidth. - Trillium is a component of AI Hypercomputer, Google's architecture combining hardware, open software (JAX, PyTorch, TensorFlow, XLA compiler) and flexible consumption models. - AI21 Labs, maker of the Jamba models, is quoted as a customer. - Optimizations of the XLA compiler and frameworks, as well as host-offloading to the host's DRAM, complement the HBM memory. ### Why it matters Choosing an AI accelerator has become an architecture decision: TPU, NVIDIA GPU or other clouds' in-house chips. Trillium shows that Google sells the infrastructure that trained its own flagship model. For customers, software portability (JAX, PyTorch/XLA) and availability in the target region matter as much as the performance figures, which remain the vendor's own. ### Takeaway The sixth-generation TPU is open to all: Google Cloud sells what trained Gemini 2.0.

Tags : Google Cloud, Puces, Hyperscaler, IA