Hardware / GPU servers / HGX H200
Spec sheet · Hopper · 2024
HGX H200 SXM: 8-GPU Server System
141 GB HBM3e per GPU. The memory upgrade over H100 that wins on long-context inference and large-model serving.
Full specifications
Compute
- GPUs
- 8× NVIDIA H200 SXM
- FP8 (system)
- ~16 PFLOPS dense
- BF16 (system)
- ~8 PFLOPS dense
Memory
- GPU memory
- 141 GB HBM3e per GPU · 1.1 TB total
- Memory bandwidth
- 4.8 TB/s per GPU · 38.4 TB/s total
Interconnect
- GPU interconnect
- NVLink 4 · 900 GB/s per GPU · NVSwitch
- Network fabric
- 8× ConnectX-7 400G NDR InfiniBand
Power and cooling
- GPU TDP
- 700 W per GPU
- System power draw
- 10.2 kW max (DGX H200)
Platform
- Host platform
- 2× Xeon SP or EPYC · 2 TB DDR5 · PCIe Gen 5
- Common chassis
- Supermicro SYS-821GE-TNHR2 · Dell XE9680
- Form factor
- 6U–8U air-cooled, same chassis as H100
- Architecture
- Hopper
- First shipped
- 2024
Power, cooling and rack density
| Cooling | Per node | Per rack | Rack power |
|---|---|---|---|
| Air | 10.2 kW | 4 nodes | 41 kW |
| Liquid | 10.2 kW | 8 nodes | 82 kW |
A 32-node cluster (256 GPUs) needs about 0.5 MW of facility power: a colocation cage or suite. Size your own cluster in the site planner →
What one node can serve
Largest open models that fit on eight GPUs at FP8 with an 8K context, with estimated output throughput.
- DeepSeek-V4.1 Flash7,318 tok/s
- GLM-5.36,960 tok/s
- gpt-oss-120b30K tok/s
- Qwen3.6 27B61K tok/s
Cost per million tokens in the inference calculator →
How it compares
| System | Architecture | Memory / GPU | Bandwidth | FP8 dense | Max power | Cooling |
|---|---|---|---|---|---|---|
| HGX B300 | Blackwell Ultra | 288 GB | 8 TB/s | 4.5 PF | 15 kW | Liquid / Air |
| HGX B200 | Blackwell | 180 GB | 8 TB/s | 4.5 PF | 14.3 kW | Air / Liquid |
| HGX H200 | Hopper | 141 GB | 4.8 TB/s | 2.0 PF | 10.2 kW | Air / Liquid |
| HGX H100 | Hopper | 80 GB | 3.35 TB/s | 2.0 PF | 10.2 kW | Air / Liquid |
| HGX A100 80GB | Ampere | 80 GB | 2 TB/s | — | 6.5 kW | Air |
| RTX Pro 6000 Blackwell | Blackwell | 96 GB | 1.8 TB/s | 1.0 PF | 6.5 kW | Air |
How it connects to the cluster
Each GPU gets its own network card and its own port on the leaf switch, one rail per GPU. These are the parts on that path.
On this path: ConnectX-7 · 400G OSFP DR4 · 800G OSFP 2×DR4 · Quantum-2 QM9700 · Spectrum-4 SN5600
Alternatives to consider
What to verify before you buy
- Same checks as H100: GPU ECC error history is critical
- HBM3e stacking defects: confirm vendor ran full memory stress test
- Neocloud decommission lots: verify service records and runtime hours
- Burn-in report with stress test results (at least 24h run)
Export controls
H200 carries the same EAR export control classification as H100. End-user screening required for all cross-border transactions.
Frequently asked questions
How much power does an HGX H200 draw?
Up to 10.2 kW per 8-GPU HGX node at maximum rated power. Plan site power for that peak: training runs close to it. Inference typically averages 55–65% of it.
Is the HGX H200 air-cooled or liquid-cooled?
Both. HGX H200 systems ship air or direct-to-chip liquid-cooled. Air racks typically hold 4 nodes (about 41 kW). Direct-to-chip liquid racks typically hold 8 nodes (about 82 kW).
How much GPU memory does an HGX H200 have?
141 GB per GPU, 1,128 GB across eight, at 4.8 TB/s per GPU.
What models can one HGX H200 node serve?
At FP8 with an 8K context, the largest open models that fit on eight GPUs include DeepSeek-V4.1 Flash, GLM-5.3, gpt-oss-120b, Qwen3.6 27B. The inference calculator shows throughput and cost for each.
What kind of site does an HGX H200 cluster need?
A 32-node cluster (256 GPUs) needs roughly 0.5 MW of facility power with air cooling: a colocation cage or suite.
What comes after the HGX H200?
The HGX B200. The HGX H200 is Hopper architecture, first shipped in 2024.
Sources: NVIDIA H200. Power is the maximum rated system draw; throughput and site figures are planning estimates from the same models as the Rack Rate tools.
Building with HGX H200?
Send us the cluster spec. We source the full bill of materials, and can finance it.