Energy efficiency across the AI datacenter stack
What does the 2023-2026 literature establish about improving energy efficiency across the AI datacenter stack - from GPU power management and workload scheduling to grid-level coordination - and how strong is the evidence for each lever?
AI datacenters are among the fastest-growing electricity consumers of the 2020s, and the 2023-2026 literature offers a stack of levers to improve their efficiency: GPU power capping and frequency control, energy- and carbon-aware scheduling, LLM serving configuration, cooling control, and grid-interactive operation. Evidence is strongest at the device and workload levels (measured savings of roughly 15-50% on specific workloads), moderate for scheduling (mostly simulation), and thinnest at grid level, where a single production deployment carries much of the weight. The key caveat: nearly every lever is workload- and hardware-generation-specific - power capping that saves ~23% energy on one model hurts SLOs on another - so none generalises without measurement.
Updated 7 Aug 202693 sources2023–2026Deep18 min read
GPU power capping · tokens per watt · carbon-aware scheduling · data center demand response · LLM inference energy · digital twins · MaxQ · power smoothing