“NVIDIA hardware and software innovations are creating a new class of AI supercomputers,” said Ian Buck, vice president of the company’s high performance computing and hyperscale data center business, in a special address at the conference.
Some of the systems will pack memory-enhanced NVIDIA Hopper accelerators, others a new NVIDIA Grace Hopper systems architecture. All will use the expanded parallelism to run a full stack of accelerated software for generative AI, HPC and hybrid quantum computing.
Buck described the new NVIDIA HGX H200 as “the world’s leading AI computing platform.”
The NVIDIA Hopper architecture delivers an unprecedented performance leap over its predecessor and continues to raise the bar through ongoing software enhancements with H100, including the recent release of powerful open-source libraries like NVIDIA TensorRT™-LLM.
The introduction of H200 will lead to further performance leaps, including nearly doubling inference speed on Llama 2, a 70 billion-parameter LLM, compared to the H100. Additional performance leadership and improvements with H200 are expected with future software updates.
In terms of benefits for AI, NVIDIA says the HGX H200 doubles inference speed on Llama 2, a 70 billion-parameter LLM, compared to the H100. It’ll be available in 4- and 8-way configurations that are compatible with both the software and hardware in H100 systems. It’ll work in every type of data center, (on-premises, cloud, hybrid-cloud and edge), and be deployed by Amazon Web Services, Google Cloud, Microsoft Azure and Oracle Cloud Infrastructure, among others. It’s set to arrive in Q2 2024.
NVIDIA’s other key product is the GH200 Grace Hopper “superchip” that marries the HGX H200 GPU and Arm-based NVIDIA Grace CPU using the company’s NVLink-C2C interlink. It’s designed for supercomputers to allow “scientists and researchers to tackle the world’s most challenging problems by accelerating complex AI and HPC applications running terabytes of data,” NVIDIA wrote.
The GH200 will be used in “40+ AI supercomputers across global research centers, system makers and cloud providers,” the company said, including from Dell, Eviden, Hewlett Packard Enterprise (HPE), Lenovo, QCT and Supermicro. Notable among those is HPE’s Cray EX2500 supercomputers that will use quad GH200s, scaling up to tens of thousands of Grace Hopper Superchip nodes.
The new technologies will be key for NVIDIA, which now makes most of its revenue from the AI and data center segments. Last quarter the company saw a record $10.32 billion in revenue in that area alone (out of $13.51 billion total revenue), up 171 percent from a year ago. It no doubt hopes the new GPU and superchip will help continue that trend. Just last week, NVIDIA broke its own AI training benchmark record using older H100 technology, so its new tech should help it extend that lead over rivals in a sector it already dominates.