AMD Helios: 2.9 EFLOPS, 31 TB of HBM4, and the Rack-Scale Bet Against NVIDIA

by Geth Author

AMD's ambitions in the AI infrastructure market took a concrete step forward on August 21st with the official launch of Helios — a rack-scale AI system designed to take on NVIDIA's NVL-series racks at every level: compute density, memory bandwidth, and scale-out networking. The numbers are staggering, and the customer list already includes Microsoft Azure and OpenAI.

What's Inside a Helios Rack

A single Helios rack packs 72 liquid-cooled AMD Instinct MI455X GPUs, built on the CDNA 5 architecture and equipped with HBM4 memory. These are joined by 6th Gen AMD EPYC 9006 (Venice) CPUs, AMD Pensando networking silicon, and the full ROCm software stack — all from a single vendor, in a single rack, under a single support contract. The system fits into 18 compute trays and 6 switch trays in a 4OU ORW rack.

The specs that matter most for AI workloads:

  • 2.9 EFLOPS of AI compute per rack (FP8)
  • 31 TB of total HBM4 memory
  • 1.7 PB/s of total HBM4 bandwidth
  • 260 TB/s of intra-rack GPU-to-GPU bandwidth
  • 43 TB/s of server-to-server bandwidth via Pensando networking

NVIDIA Is the Clear Target

AMD has been building toward this moment for years. The MI300X was competitive in certain inference workloads; Helios is AMD's answer to NVIDIA's GB200 NVL72 at the rack scale. The architecture mirrors the thermal approach NVIDIA has adopted — liquid cooling, extreme density — while betting that a single-vendor integrated stack simplifies procurement and support in ways a best-of-breed ecosystem approach cannot.

Who's Actually Buying It

AMD announced that Helios deployments are beginning in H2 2026. Microsoft Azure and OpenAI are both on record with Helios commitments — which is the most meaningful signal possible. OpenAI running workloads on AMD hardware signals that ROCm has reached a point where production AI training and inference on AMD is viable at scale. If that holds up in practice, the competitive dynamics of the AI accelerator market shift meaningfully.

The ROCm Question

Hardware specs only matter if the software works. AMD's ROCm ecosystem has historically trailed CUDA in tooling maturity, framework compatibility, and debugging experience. The Helios launch coincides with what AMD describes as a substantially improved ROCm release — but the proof will be in sustained production deployments. If Azure and OpenAI report smooth operations over the next two quarters, AMD's AI narrative changes substantially. If they hit friction, the specs become a footnote.

Helios is available now in production. More details on the system architecture can be found at AMD's official blog.