Google introduced two new Tensor Processing Units this week—the TPU 8tx and TPU 8i Max—extending its eighth-generation chip line to address two specific cloud economics problems: the ability to train on sensitive data without exposing it, and the memory ceiling that constrains inference on large models.
The TPU 8tx, developed with AMD, builds on Google's existing TPU 8t to add confidential computing at the silicon level. The chip isolates model weights and training data in hardware-protected memory even during active computation, letting customers train on proprietary datasets without moving data between devices. Google also embedded a custom inference network that allows simultaneous training and reasoning—enabling on-the-fly model distillation without pipeline stalls.
The TPU 8i Max pairs two inference-focused TPU 8i chips, doubling memory capacity and bandwidth in a single package. The economics here are straightforward: larger in-context windows allow bigger batch sizes and longer reasoning chains, expanding the addressable workload mix a customer can run on a single instance.
Both moves directly target cloud margin architecture. By controlling silicon design, Google optimizes for its own software stack and workload patterns—something an external vendor cannot do. That vertical integration tightens the competitive moat against AWS and Azure, who face the same margin pressure from AI inference at scale.
The deeper pattern: Microsoft, Amazon, OpenAI, Anthropic, and now Google are all building custom silicon. The economics are compelling—hyperscalers can amortize R&D across massive internal fleets, negotiate better manufacturing terms, and tune silicon to their specific inference and training regimes. The era of buying commodity GPUs and calling it a cloud advantage is ending.
