Hardware / GPU servers / HGX H100

Spec sheet · Hopper · 2022

HGX H100 SXM5: 8-GPU Server System

The workhorse Hopper node: eight H100 SXM5 GPUs on NVLink 4, 640 GB of HBM3, and the most mature software stack of any HGX system.

Memory per GPU80 GB640 GB per node
Bandwidth per GPU3.35 TB/sHBM
FP8 dense per GPU2.0 PFNo native FP4
Max power10.2 kWper 8-GPU node
HGX H100 SXM5: 8-GPU Server System

Full specifications

Compute

GPUs
8× NVIDIA H100 SXM5
FP8 (system)
~16 PFLOPS dense
BF16 (system)
~8 PFLOPS dense

Memory

GPU memory
80 GB HBM3 per GPU · 640 GB total
Memory bandwidth
3.35 TB/s per GPU · 26.8 TB/s total

Interconnect

GPU interconnect
NVLink 4 · 900 GB/s per GPU · NVSwitch
Network fabric
8× ConnectX-7 400G NDR InfiniBand

Power and cooling

GPU TDP
700 W per GPU
System power draw
10.2 kW max (DGX H100)

Platform

Host platform
2× Xeon SP or EPYC · 1–2 TB DDR5 · PCIe Gen 5
Common chassis
Supermicro SYS-821GE · Dell XE9680 · HPE XD670
Form factor
6U–8U air-cooled · liquid-cooled variants exist
Architecture
Hopper
First shipped
2022

Power, cooling and rack density

CoolingPer nodePer rackRack power
Air10.2 kW4 nodes41 kW
Liquid10.2 kW8 nodes82 kW

A 32-node cluster (256 GPUs) needs about 0.5 MW of facility power: a colocation cage or suite. Size your own cluster in the site planner →

What one node can serve

Largest open models that fit on eight GPUs at FP8 with an 8K context, with estimated output throughput.

  • gpt-oss-120b21K tok/s
  • Qwen3.6 27B43K tok/s
  • gpt-oss-20b57K tok/s

Cost per million tokens in the inference calculator →

How it compares

SystemArchitectureMemory / GPUBandwidthFP8 denseMax powerCooling
HGX B300Blackwell Ultra288 GB8 TB/s4.5 PF15 kWLiquid / Air
HGX B200Blackwell180 GB8 TB/s4.5 PF14.3 kWAir / Liquid
HGX H200Hopper141 GB4.8 TB/s2.0 PF10.2 kWAir / Liquid
HGX H100Hopper80 GB3.35 TB/s2.0 PF10.2 kWAir / Liquid
HGX A100 80GBAmpere80 GB2 TB/s—6.5 kWAir
RTX Pro 6000 BlackwellBlackwell96 GB1.8 TB/s1.0 PF6.5 kWAir

How it connects to the cluster

Each GPU gets its own network card and its own port on the leaf switch, one rail per GPU. These are the parts on that path.

8× H100 SXM58× ConnectX-7 · 400G400G OSFP DR4, flat-top800G OSFP 2×DR4, finnedLeaf: Quantum-2 or Spectrum-4

On this path: ConnectX-7 · 400G OSFP DR4 · 800G OSFP 2×DR4 · Quantum-2 QM9700 · Spectrum-4 SN5600

Alternatives to consider

Consider H200 if you're memory-bound141 GB HBM3e vs 80 GB: long-context inference and large model serving often justify the step up.Consider B200 if you're building for 3+ yearsBlackwell wins on performance per watt and roadmap headroom, at higher power per node.

What to verify before you buy

  • GPU ECC error history: high uncorrectable errors are a hard no
  • BMC / iDRAC event logs for thermal events or power faults
  • NVLink and baseboard health from burn-in reports
  • Firmware versions and attestation state
  • OEM service tag status: is warranty alive, transferred, or void?
  • Provenance and clean title: no export hold or lien
  • Burn-in report with stress test results (at least 24h run)

Export controls

H100-class accelerators are controlled under US EAR export regulations regardless of whether they are new or used. Cross-border resale requires end-user screening and may require an export license for certain destinations. Factor compliance into any international deal timeline.

Frequently asked questions

How much power does an HGX H100 draw?

Up to 10.2 kW per 8-GPU HGX node at maximum rated power. Plan site power for that peak: training runs close to it. Inference typically averages 55–65% of it.

Is the HGX H100 air-cooled or liquid-cooled?

Both. HGX H100 systems ship air or direct-to-chip liquid-cooled. Air racks typically hold 4 nodes (about 41 kW). Direct-to-chip liquid racks typically hold 8 nodes (about 82 kW).

How much GPU memory does an HGX H100 have?

80 GB per GPU, 640 GB across eight, at 3.35 TB/s per GPU.

What models can one HGX H100 node serve?

At FP8 with an 8K context, the largest open models that fit on eight GPUs include gpt-oss-120b, Qwen3.6 27B, gpt-oss-20b. The inference calculator shows throughput and cost for each.

What kind of site does an HGX H100 cluster need?

A 32-node cluster (256 GPUs) needs roughly 0.5 MW of facility power with air cooling: a colocation cage or suite.

What comes after the HGX H100?

The HGX H200. The HGX H100 is Hopper architecture, first shipped in 2022.

Sources: NVIDIA H100 · NVIDIA DGX H100 user guide. Power is the maximum rated system draw; throughput and site figures are planning estimates from the same models as the Rack Rate tools.

Building with HGX H100?

Send us the cluster spec. We source the full bill of materials, and can finance it.

Start a buildExplore financing