NVIDIA Infuses Grace Hopper Superchip With HBM3e To Supercharge AI Data Center Workloads
It's a little bit unusual, to tell the truth. The Grace Hopper Superchip isn't a new product in and of itself; NVIDIA's been selling them since earlier this year. It combines a 72-core Grace CPU (based on ARM Neoverse V2 processor cores) and its 500GB of LPDDR5X memory with an H100 Hopper GPU to provide Super Saiyan God levels of compute on a single PCB.
The original Grace Hopper superchip came with up to 96GB of HBM3 for its Hopper GPU, but the new version instead packs on 141 GB of the latest and greatest HBM3e RAM. 141GB is a weird number, but we expect that it originates from five 32GB stacks (160GB) with ECC enabled, reducing the capacity to 7/8ths of its physical size. That gives a capacity of roughly 141 GB if you round things off with SI units instead of binary (Gibibytes) units. Whatever the case, it is an incredible amount of memory for a GPU.
Besides the extra capacity, NVIDIA says that the HBM3e version of the Hopper GPU offers a 50% bandwidth improvement, too. That gives the massive parallel processor some five terabytes per second of memory bandwidth, which is a number that boggles the mind. The H200 GPU and Grace CPU talk to each other over a 900 GB/sec coherent NVLink interconnect, and each Superchip can be linked to additional chips in the same way, allowing them to act fundamentally like a single unit.
Interestingly, despite the "GH200" name, NVIDIA hasn't talked about any changes to the GPU die itself. Historically, NVIDIA's model numbers categorically denoted changes to the product and architecture, with the rare "2" in the first numeric position indicating a full architectural revision such as with the "Maxwell 2" architecture that underpinned the GeForce 900 series. All of those parts were code-named "GM2xx". It's possible that the Hopper chip itself has seen a silicon revision with this release, but if so, NVIDIA isn't talking about it.
If you're someone whose work involves systems at the exaflop scale, we're very pleased that you're reading HotHardware for your GPU news. However, you'll have to wait before you can get your hands on the new GH200. NVIDIA says that "leading system manufacturers" should have machines sporting the new hardware in Q2 of 2024, with pricing no doubt being in the "if you have to ask..." territory.
