How much power does a GPU rack actually need?
H100, H200 and B200 deployments fail more often on power and cooling than on floor space. Here is how to translate NVIDIA’s published GPU TDP into a facility conversation that does not undersize the rack.
Teams shopping for high-density colocation often start with cabinet count. For modern GPU clusters, the binding constraint is usually usable kW per rack, cooling method, and how fast the provider can energize that power. Floor space is the easy part.
This note sticks to published hardware power figures and the commercial questions those figures force. It does not invent market pricing or promise delivery timelines.
Published GPU power (TDP / TGP)
NVIDIA publishes maximum thermal design power for each product. Use the SXM (board) figures for HGX-style training and inference nodes; PCIe / NVL cards are lower but still dense when you pack eight of them.
| GPU | Form factor | Max GPU power | GPU memory |
|---|---|---|---|
| H100 | SXM | Up to 700 W (configurable) | 80 GB HBM3 |
| H100 | NVL / PCIe | 350–400 W (configurable) | 94 GB |
| H200 | SXM | Up to 700 W (configurable) | 141 GB HBM3e |
| H200 | NVL | Up to 600 W (configurable) | 141 GB HBM3e |
| B200 | SXM | 1,000 W TGP | 180 GB HBM3e |
An 8-GPU HGX H100 or H200 board alone is up to about 5.6 kW of GPU silicon. An 8-GPU B200 board alone is about 8 kW of GPU silicon. That is before CPUs, memory, local storage, NICs, fans or pumps, and power-supply overhead.
From GPU watts to node and rack draw
A practical planning sequence:
- Sum the GPU maxima for the node (for example 8 × 700 W = 5.6 kW on H100/H200 SXM, or 8 × 1,000 W = 8 kW on B200).
- Add the rest of the node: dual-socket CPUs, DRAM, NVMe, BlueField or ConnectX adapters, and chassis overhead. Exact totals depend on the OEM bill of materials; treat GPU watts as the floor, not the ceiling.
- Multiply by nodes per rack, then leave headroom for diversity, future cards, and the fact that providers sell committed kW, not theoretical peaks.
- Separate nameplate from operating load in the conversation, but size cooling and feeder capacity against the committed figure you will contract.
Legacy retail colocation was often sold in the mid–single-digit kW per rack. Multi-GPU AI nodes push many deployments into tens of kW per rack, and dense liquid-cooled rows go higher still. If a provider’s standard product tops out below your per-rack requirement, you need a high-density product, a private cage with custom power, or a different facility entirely. RightCapacity’s live off-market capacity page is where we surface GPU-ready options that are not always marketed publicly.
| Planning question | Why it matters |
|---|---|
| kW per rack (committed) | Sets how many nodes fit before you buy another cabinet or a private cage. |
| Redundancy (N, N+1, 2N) | Changes feeder count, UPS topology and usable capacity after derating. |
| Voltage and plug type | High-density rows often need different PDUs and receptacle standards than legacy cabinets. |
| Growth reserve | Adding GPUs later is useless if the rack and the cooling loop are already maxed. |
When air stops and liquid starts
Air-cooled halls still handle many inference and lighter training footprints, especially with careful airflow and lower GPU counts per rack. As node TDP climbs toward B200-class boards, facility designs increasingly assume rear-door heat exchangers or direct liquid cooling to the cold plate.
What to confirm in writing:
- Whether the hall is qualified for your OEM’s thermal envelope (inlet temperature, liquid supply temperature, flow rate).
- Who owns the CDU, manifolds, hoses and leak detection: provider, customer, or shared.
- Whether liquid is available on day one in that row, or only as a forward fit-out.
- How maintenance windows and fluid quality are handled in the MSA or an exhibit.
Cooling method should sit in the same brief as power and cross-connects when you use Find capacity or talk through an AI & GPU infrastructure requirement. Hardware-only sourcing without a facility path is how projects end up with GPUs and nowhere to land them. If you still need the systems themselves, pair the facility search with GPU sourcing or short-term GPU rentals.
What to ask a colocation provider
- What is the maximum committed kW for the specific cabinet or cage on offer, not the building’s marketing “up to” figure?
- Is that power available now, on a sublease, or as forward capacity waiting on utility or fit-out work?
- What is the cooling method for that exact location, and has the provider already deployed the same GPU generation there?
- Which carriers and cloud on-ramps sit in the meet-me room, and what do cross-connect lead times look like in practice?
- Can you grow in-row without a full redesign of power and liquid?
In tight markets such as Ashburn, San Jose, Dallas and Chicago, the honest answers to those questions matter more than brochure density ratings. Sublease and forward options often clear first when public inventory looks empty. Operators with spare high-density cabinets can submit capacity; buyers can sign up for capacity alerts.
FAQ for GPU colocation buyers
Is GPU TDP the same as the power I should contract in a colo cabinet?
No. Published GPU TDP or TGP is the silicon’s thermal design limit. Facility contracts are written around committed IT load at the rack or cage, which also includes CPUs, memory, storage, networking and power-supply overhead. Use GPU watts as a floor, then size the node and rack from the full bill of materials.
How much GPU power does an 8-GPU H100/H200 or B200 node represent?
Per NVIDIA’s published maxima, eight H100 or H200 SXM GPUs are up to about 5.6 kW of GPU silicon (8 × 700 W). Eight B200 SXM GPUs are about 8 kW of GPU silicon (8 × 1,000 W). That figure still excludes the rest of the server.
When should liquid cooling enter the facility brief?
When node density and inlet-temperature limits push past what the hall’s air product can reject for your OEM envelope. Confirm whether rear-door heat exchangers or direct-to-chip liquid are qualified in that exact row, who owns the CDU and manifolds, and whether liquid is available now or only as a forward fit-out.
Takeaways
- Size from published GPU TDP/TGP, then add the rest of the node. Never treat GPU watts as the rack total.
- 8× H100/H200 SXM is up to ~5.6 kW of GPU silicon; 8× B200 is ~8 kW of GPU silicon, before CPUs, storage and networking.
- Contract committed kW, redundancy and cooling method for the specific row or cage, not a campus “up to” number.
- If public listings do not show the density you need, look at off-market, sublease and forward capacity before changing markets.
Need GPU-ready power and cooling, not just empty cabinets? Share the node count, GPU model and target market.
Discuss your deployment ↗- NVIDIA H100 Tensor Core GPU datasheet (SXM max TDP up to 700 W configurable; NVL 350–400 W).
- NVIDIA H200 product specifications (SXM up to 700 W; NVL up to 600 W).
- Lenovo ThinkSystem NVIDIA HGX B200 180GB product guide (1,000 W TGP).
- NVIDIA HGX AI Factory reference (H100 / H200 / B200 8-GPU baseboards).