GPU servers
NVIDIA HGX H200 8-GPU server
8 x H200 SXM, 1,128 GB of HBM3e in one node.
The HGX H200 platform puts eight H200 SXM GPUs on one NVLink baseboard. Each GPU carries 141 GB of HBM3e at 4.8 TB/s, so a single node holds 1,128 GB of GPU memory: enough to serve 70B-class models at full precision or fine-tune larger ones without sharding across nodes.
Built for
- Large-model inference
- Fine-tuning 70B+ models
- Research clusters
Questions
H200 has 76% more memory per GPU (141 GB against 80 GB) and 43% more bandwidth (4.8 TB/s against 3.35 TB/s) on the same Hopper architecture. If your models or batch sizes are memory-bound, H200 is the better buy; for smaller models a used H100 node is the cheaper entry.