Smoothing the power curve of LLM training and inference

Why this question

Power draw shapes what an AI datacenter can build. Facilities are sized to peaks and ramps, not averages: transformers, UPS capacity, cooling, and grid interconnection agreements are all dimensioned so that the worst 5-minute power excursion fits, and grid operators increasingly condition new AI capacity on demand-response behaviour with explicit ramp limits ⁴⁴. Energy efficiency (tokens per watt) has become a headline metric, but a workload can be energy-efficient on average and still be expensive to host if its instantaneous power is spiky — spikes force over-provisioning, and rapid ramps force reserve capacity or penalties ⁴³. So the operational question for anyone running or designing LLM infrastructure is not only “how many joules per token” but “can the joules be delivered on a flatter curve, and what does flattening cost?”

The literature has historically answered the first question far better than the second. A mature device-level literature on GPU DVFS and power capping (2007–2019, HPC and graphics workloads) established the energy–throughput trade-off frontier; a fast-growing 2023–2026 wave has re-run those experiments on LLM training and inference and discovered that the LLM workload’s phase structure — compute-bound prefill versus memory-bound decode — changes which levers work at all. A third, very recent thread connects GPU scheduling to grid behaviour explicitly: ramp rates, coincidence factors, and demand-response contracts. This review sits between the site’s tokens-per-watt review (how efficiency is measured), its kernel-optimisation review (kernel performance), and its model-to-grid review (facility- and grid-level efficiency): it covers the GPU- and workload-level techniques that shape the power waveform itself.

Scope and methods

Question. Which kernel-, frequency-, and scheduler-level techniques reduce power ripples — spikes, ramps, and variance — during LLM training and inference, and what do they cost in energy and throughput?

Inclusion criteria. Peer-reviewed or preprint work measuring or modelling GPU power/energy for deep-learning training or inference, with emphasis on (a) kernel- or phase-level frequency/voltage control, (b) power capping, (c) workload scheduling that shapes power draw, (d) energy–throughput trade-offs at device-to-cluster scale, and (e) measurement methodology for power patterns. Years 2007–2026, any language (English-language records only in practice).

Exclusion criteria. Facility-level cooling, PUE, and carbon-accounting work (covered in the model-to-grid review); pure kernel performance work without a power/energy component (covered in the kernel reviews); CPU-only DVFS; grid/energy-storage work that does not touch compute workloads.

Search. 44 literal query strings across OpenAlex, the arXiv API, Crossref, and Semantic Scholar (best-effort; 429-blocked for most of the session), plus DBLP and arXiv abs-page resolution for landmark titles (Zeus, Perseus, kernel-level DVFS). Rung-0 reuse of 36 verified records from prior reviews on this site (tokens-per-watt, model-to-grid, kernel-training, interconnects, DiLoCo). Screening: 1,265 unique candidates pooled and deduplicated → 91 selected on title/abstract → 89 included (2 dropped when no abstract was retrievable anywhere: a Springer DVFS chapter and a Research Square preprint). 39 of 89 sources were read in full text; the remainder are abstract-only and are used only for claims their abstracts support. Every DOI was verified against Crossref or the arXiv abs page, and every load-bearing number cited below was re-grepped against the retrieved text (see the “phantom inference Perseus” note in Gaps for one widely-circulated title that does not exist as a distinct publication).

The landscape

Three literatures meet here, with different maturities. The oldest is device-level GPU energy management: DVFS experiments and surveys from the HPC/graphics era established that core-frequency scaling can save roughly 8–26% of energy on DNN workloads at single-digit performance cost ⁶⁹⁵³, that core and memory clocks interact differently across architectures and applications ⁵³, and that GPUs carry a voltage guardband of about 20% that undervolting can reclaim in part ³⁹. A 2014 survey already catalogued the full toolbox — DVFS, clock gating, power capping, kernel-level techniques — for GPU energy efficiency ⁵⁴, and a 2025 ACL survey organises the LLM-inference-specific energy literature and its optimisation levers ¹⁶. These results were mostly single-GPU and small-model, but they established the vocabulary and the measurement practice.

The second literature is LLM-era systems: from roughly 2023, power-aware serving systems (frequency governors, cap controllers, phase-split architectures) and training systems (energy-bloat removal, power-aware collectives) began publishing measured energy numbers for transformer workloads at the scale that matters. This wave is strikingly concentrated in 2025–2026, much of it preprint-only, and much of it from a small number of research groups (the Michigan ml.energy lineage — Zeus, Perseus, Kareus ⁸⁵⁹⁸¹; the e-Energy/HPCA serving systems lineage ⁶⁸³²⁸⁶). The third literature is grid-side: power-electronics and power-systems work treating AI datacenters as flexible, ramp-constrained loads ⁴⁴⁸⁰. Its vocabulary — ramp rate, coincidence factor, reserve capacity — is only now entering the systems literature through papers like “Smoothing the Ramp, Not the Peak” ⁴³.

Methodologically the field leans on single-node measurement (a handful of A100/H100/H200 GPUs), emulation for scale (Perseus’s 1,024-GPU claims are emulated ⁹), and simulation for cluster and grid effects. Field studies at production scale are rare; the notable exception is a 130 kW real deployment used to demonstrate grid-responsive GPU-cluster control ⁸⁰. Benchmarks that capture power patterns rather than total energy are only now appearing ⁵⁹⁷¹³.

Theme 1 — Where the ripples come from: phases, memory-bound decode, and static energy

The single most important structural fact in this literature is that an LLM inference pass is two different workloads. Prefill is compute- and capacity-bound, drives GPUs toward their thermal/power ceiling, and is sensitive to frequency; decode is memory-bandwidth-bound, uses a small fraction of the GPU’s compute and power headroom, and is nearly insensitive to compute frequency ^63865048. The imbalance is large: across workloads and GPUs, decode dominates wall-clock inference time (77–91% in one characterisation ⁵⁰) while drawing far less than the GPU’s rated power — 137–300 W on a 700 W H200 in one measurement campaign spanning four attention architectures (GQA, MLA, Gated DeltaNet, Mamba2) ⁴⁸. Phase-splitting systems exploit exactly this asymmetry, putting the two phases on different hardware so that each machine sees a flatter, more utilising load ⁶³.

Ripples therefore arise at several levels. Within a request, the prefill→decode transition is a power cliff: power falls as compute yields to memory traffic. At the server level, batching and scheduling policy decides how many prefills land in the same instant; chunked prefill (splitting a long prompt’s prefill into smaller scheduled steps — a latency technique already deployed by default in production serving) spreads that work, and measurements with real GPU power traces show it is a ramp-rate regulator: it cuts the mean power ramp by ~35% (from ~46 to ~30 W/s) under heavy load while leaving the peak nearly identical, because the peak is set by what a saturated machine draws regardless of chunking ⁴³. At the training level, ripples come from synchronisation: pipeline-stage imbalance makes non-critical stages run needlessly fast (Perseus’s “intrinsic energy bloat”) ⁹; collective communication during overlap inflates power relative to compute-only phases ³⁸; and checkpoints add discrete energy lumps whose size depends on processor power states and I/O configuration ⁵⁵.

A final, easily missed component is static and idle energy. Measurement studies of production inference found that a large share of GPU energy is drawn outside actual compute — model loading, engine setup, and standby — so engine choice and idle management matter as much as kernel efficiency ⁵⁸⁴⁶. Idle GPUs waiting at synchronisation barriers in geo-distributed training waste energy at a rate that power-aware synchronisation can cut substantially ⁵². Kernel-scheduling order and concurrency level measurably change energy even on embedded GPUs ⁷⁷. Any power-flattening strategy has to decide whether it is shaping the compute phases or paying for the valleys between them.

Theme 2 — The frequency lever: DVFS sweet spots, granularity, and switching cost

DVFS is the canonical power-shaping knob, and the LLM-era measurements agree with the older HPC results on the shape of the trade-off: energy-versus-frequency curves are U-shaped with a sweet spot well below maximum frequency, because near the top of the clock range each additional MHz buys little throughput for a lot of power, while far below it the job simply runs longer and fixed overheads dominate ⁸⁶⁶²⁵³. Quantitatively: cutting an A100 from 1,410 MHz to 1,005 MHz for decode roughly halves decode energy for a ~20% latency increase — the point where the U-curve bottoms out — in VoltanaLLM’s measurements ⁸⁶; across a broad GPU-frequency sweep on inference workloads, dropping from 2,842 MHz to 180 MHz saves ~42% of energy for only 1–6% latency increase because decode is memory-bound ⁵⁰; and embedded/edge LLM DVFS studies find optima in the 510–816 MHz band with average savings of ~13% (up to ~27%) ⁶².

Two mechanisms make the sweet spot phase-dependent rather than global. First, the two clock domains matter differently: core (SM) frequency governs compute-bound kernels, memory frequency governs memory-bound ones, and the correct pairing is architecture- and application-specific ⁵³. Second, the sensitivity differs between prefill and decode: prefill loses throughput almost linearly as frequency drops, while decode sustains throughput down to ~0.57× nominal frequency ⁴². Operator-level granularity goes further: within a layer, Attention and FFN kernels have different energy-optimal frequencies, which AFlex exploits by disaggregating and scaling the two operator types separately (up to ~49% energy-per-token reduction versus state-of-the-art baselines) ²². Decode-stage DVFS analyses reach the same conclusion from the measurement side: prefill and decode respond differently to the same frequency settings, so stage-aware policies are needed ⁷⁹.

How finely can the lever be moved? Predictive fine-grain DVFS rests on nanosecond-regime voltage/frequency transition times on modern GPUs ⁶, but measured end-to-end switching latency is much coarser and strongly asymmetric — under ~25 ms on A100 for most frequency pairs, with far slower regions on other GPUs ⁷⁴. Switching cost is therefore a real constraint on how often a governor should change clocks, and switching-aware online optimisation (bandit-style controllers that budget for transitions) measurably beats naive per-interval control ⁸². Empirical studies of GPU frequency behaviour under ML workloads also show that firmware’s automatic clock management (boost behaviour) is itself a source of power variance that manual clock control removes ³⁷⁴⁸. At the training-kernel level, a 2026 study of GPT-3-class training on MI300X found that DVFS frequency effects were the single largest contributor to the gap between theoretical and observed performance — larger than compute-communication overlap losses or kernel-launch overhead — and that kernel-level (per-kernel) clock selection saves ~14.6% of energy at 0.6% slowdown, where pass-level policies achieve only ~2% ⁶⁷³⁶. Older work anticipated the mechanism: fusing or reordering dependent kernels changes the power profile and can lower peak power without performance loss ⁷⁶, and kernel-aware model-predictive frequency control beat reactive schemes by ~25% energy at <2% performance cost as early as 2017 ⁴⁹.

The practical controllers that operationalise all this are the subject of Theme 5, but the underlying result is consistent enough to state plainly: for compute-bound phases, running GPUs at or near maximum frequency is measurably wasteful; the optimum is typically 60–80% of the clock range, and finding it online costs little ²¹⁷⁷⁸. Choosing that optimum well depends on power models that track DVFS behaviour: scaling-factor models reach 2.4–4.6% average power-prediction error across four GPU generations ²⁰, ensemble machine-learning predictors reach ~3.5% MAE ¹³, and workload classifiers cut profiling time ~89% when predicting frequency-capping behaviour of unseen applications ²⁸. Multi-GPU runtimes add placement as a second knob — DEFT’s joint task-placement-plus-DVFS saves ~15% energy on L40S-class servers at under 1.5% performance cost ⁷ — and deadline-aware DVFS schedulers show the energy gain survives real-time constraints (~15% below the best baseline with zero deadline misses) ²⁵. The voltage side of the lever is older but still active: adaptive voltage-frequency scaling below nominal on reconfigurable accelerators reached performance/energy points up to 86% better than nominal at equal accuracy ⁶⁰.

Theme 3 — Power capping: when caps engage, what they cost, and the “illusion” in decode

Power capping (setting a watt limit that the GPU enforces by throttling clocks) is the standard deployment lever because it is a single nvidia-smi-style command with no code changes. The LLM-era evidence shows its effects are strongly phase- and workload-dependent. On compute-bound inference, capping works like a softer version of DVFS with an important asymmetry: energy falls faster than throughput because the cap removes the least efficient top of the clock range. Measured on LLaMA-65B/A100 serving, a 250→175 W cap cut total energy ~23% for ~7% more time; a 150 W cap cut energy ~33–35% for ~15–22% more time ⁶⁵. Training measurements across CNN workloads find the same shape, with optimal caps typically 55–70% of TDP for compute-intensive kernels ⁶⁴³⁵³⁴. At the HPC-application level, GROMACS on A100 keeps near-peak performance under moderate caps because many workloads never reach TDP anyway — the cap only binds when the workload actually wants the power ¹.

The striking negative result comes from decode. Because memory-bound decode draws 137–300 W on a 700 W GPU, a power cap set anywhere above the actual draw never engages — the throughput loss sometimes attributed to capping is instead firmware clock throttling or measurement confounds. On H200 across four attention architectures, SM clock locking (fixing the SM clock directly, the lever actually on the critical path) Pareto-dominated power capping and recovered up to ~32% of decode energy ⁴⁸. Cap-behaviour studies across GPU generations agree: H100 sustains peak memory bandwidth even at a 200 W cap, while H200 needs ~550 W to reach peak bandwidth and visibly violates its own 200 W cap (~250 W drawn), because its HBM power is less elastic ⁷². In other words: capping is a compute-phase lever, and its apparent effect during decode phases is frequently an artefact.

Cap controllers also need to handle dynamics. Cluster-level cooperative capping of deep-learning jobs (server-level budget shared across GPUs with online frequency estimation) achieves sub-1% cap-tracking error and cuts deadline violations by ~21% ³³. Adaptive schemes balancing energy, thermals, and performance report ~13% energy and ~11% temperature reduction for ~3% runtime increase ¹¹, and energy-aware cap tuning on edge/5G AI platforms saves up to ~26% ⁵¹. Because a server’s power budget is shared, joint CPU+multi-GPU capping outperforms per-GPU capping alone for inference servers ⁴⁷, and coordinated CPU–GPU capping with performance prediction reaches ~98% prediction accuracy ⁸⁹. PID-style feedback capping through vendor power-limit interfaces — with deadbands, rate limiting, and anti-windup — yields small but real energy gains (~4%) against uncapped operation while respecting latency SLOs ⁷⁰. The unifying caveat: cap benefits saturate and can invert as targets tighten, so power-aware scheduling that steers work away from tight caps (Theme 4) generally beats pure capping at the same power budget ²¹.

Theme 4 — Shaping the schedule to flatten the curve

If DVFS and capping set the per-GPU power ceiling, scheduling decides when power is drawn. The flagship result is Perseus’s concept of energy bloat: energy consumed that does not contribute to end-to-end throughput. In pipeline-parallel training, stages off the critical path run needlessly fast (intrinsic bloat); in parallel multi-pipeline runs, stragglers — from power/thermal throttling, I/O, or failures — make every other pipeline run needlessly fast (extrinsic bloat) ⁹. Perseus pre-characterises the time–energy frontier of a training iteration (an NP-hard problem in general, solved optimally on a relaxed formulation via graph cuts) and then slows down non-critical computation so that the critical path is unchanged: up to 30% energy reduction with no throughput loss on GPT-3-175B and Bloom, in 1,024-GPU emulation, implemented as a software layer over Zeus ⁹. The mechanism is precisely power-waveform shaping: energy is moved off the moments where it cannot help and spent where it advances the critical path. Kareus extends the frontier by jointly optimising SM allocation, communication-kernel launch timing, and GPU frequency, and — crucially — accounting for static energy, which fine-grained schedules change: up to 28.3% energy reduction at the same training time ⁸¹.

Batch shaping is the inference-side analogue. Because each request carries a fixed prefill/startup energy that output tokens amortise, batching and output length are first-class energy axes: measured per-token energy on a small model falls from ~7.5 J/token to ~0.7 J/token as output length grows 10→512 tokens at batch 16 ⁷⁵. Batch size can therefore be used as a power-control knob in its own right, complementing DVFS: BatchDVFS, which binary-searches batch size under a power cap, outperforms DVFS-only and batching-only policies by large margins on throughput-under-cap ⁵⁶. Request-level energy models make these trade-offs predictable rather than reactive: analytical models estimate token-level request energy within roughly 5-27% of measured values across model sizes ⁷³, and per-request decomposition separates the fixed prefill/startup term from the per-token term ⁷⁵.

Phase disaggregation reshapes the power pattern at the fleet level. Splitwise’s observation — decode underutilises compute, prefill underutilises memory — motivated running the phases on different machines: 1.4× throughput at 20% lower cost, or 2.35× more throughput under the same power budget ⁶³. The energy-focused successors tune frequency per phase on disaggregated clusters: VoltanaLLM’s iteration-level prefill/decode frequency control saves ~36% energy under SLOs ⁸⁶; DualScale’s phase-aware placement plus DVFS cuts prefill energy up to 39% and decode energy up to 48% against a strong disaggregated baseline ⁵; ELLIE does the phase split at the edge, roughly halving energy at comparable latency ¹⁵; LLM-GreenServe schedules prefill and decode power-aware with reported active-GPU-energy reductions up to ~17% ¹⁰. One careful dissent exists: at moderate loads, disaggregation’s performance benefit is not guaranteed, and the independent frequency scaling it enables does not by itself save energy — savings come only with load conditions that actually create idle capacity ⁴¹. When demand response caps the fleet’s power envelope, phase asymmetry becomes the steering signal: PowerSlider shifts the cap toward decode (which tolerates low frequency) and away from prefill (which does not), sustaining 78% online goodput at a 30% cap reduction where the best baseline manages 48% ⁴².

The ramp result from Theme 1 completes the picture at the finest scheduling granularity: with chunked prefill, ramp rate (W/s) is a controllable, saturation-dependent quantity — a ~7% ramp reduction at light load growing to ~35% at heavy load — while peak power is not ⁴³. Ramp and peak are different degrees of freedom, and the scheduler’s lever acts on the former.

Theme 5 — SLO-aware closed-loop control for serving

A distinct sub-literature treats frequency/cap control as a real-time feedback problem with latency SLOs. The recurring architecture: a controller that observes queue/iteration state, predicts the next interval’s load, and adjusts GPU clocks or caps with hysteresis and rate limiting. Results are consistently in the 20–45% energy-savings band at single-digit SLO impact: throttLL’eM (instance count + frequency scaling with ML-projected KV-cache usage and batch size) reports up to ~44% energy reduction under SLOs versus a Triton baseline ³²; iteration-level DVFS for autoregressive decoding reports 23–46% energy gains under varying SLO constraints ³¹; GreenLLM’s phase-separated controller (length-queued routing, energy-minimal prefill clocks, 20 ms decode loop) cuts total energy up to 34% versus default DVFS with <3.5% added SLO violations ⁴⁵; AGFT’s online-RL frequency tuner saves ~44% of GPU energy for ~9% TTFT and ~7% TPOT overhead ⁸⁴; Quicktopia’s iteration-level control improves energy-delay product by up to 22% at 1–5% latency cost ⁴; EcoInfer saves up to ~25% energy at ~1.3× tokens/J with SLO attainment nearly unchanged ²³; CoFRIS coordinates frequency and compute-unit scaling for ~28% power reduction ⁸; and on-device PELM couples DVFS with speculative decoding and variable verification depth for up to ~52% energy reduction ⁸³. DynamoLLM lifts the same logic to cluster level — dynamically reconfiguring instances, tensor parallelism, and GPU frequency per pool — reporting 53% energy and 61% customer-cost reduction under SLOs in cluster experiments ⁶⁸. WattWiser adds consolidation: with multi-model multi-GPU inference, power-aware scheduling that packs work onto fewer GPUs and powers the rest down saves energy without SLO violations ²⁷. Power-aware scheduling for multi-model inference servers ²⁶ extends the same idea upward, and power-aware request reallocation across prefill/decode pools (RAPID) sustains up to 2x the SLO attainment of static assignment at peak load ³⁰; PALS shows that for MoE serving, capping plus power-aware request routing improves tokens/J by up to 26% ²¹.

The control-theoretic caution from this literature: governor behaviour on real systems can be pathological when CPU, GPU, and memory governors act independently — Android’s default governors on mobile LLM inference produce 23–40% worse latency than the optimal frequency combination at equal energy ⁸⁷. Closed-loop control needs to own the full frequency stack, not just one domain.

Theme 6 — Training systems: collectives, overlap, checkpoints, and geo-distribution

Training-side power shaping beyond Perseus/Kareus concentrates on the communication and checkpoint events that punctuate steady-state compute. Collective communication is a power event: power-aware collective scheduling (PCCL) staggers or shapes all-reduce traffic to respect power budgets, cutting collective energy ~27% and end-to-end LLM-training energy ~17% with negligible throughput impact ²⁹. Characterisation of compute–communication overlap shows why this is non-trivial: overlapping communication with compute slows the compute kernels themselves (18.9% average slowdown versus ideal), and under strict power caps the interaction between overlap and capping can inflate training time by over 2× ³⁸. Energy-aware placement of data-parallel and pipeline stages interacts with these costs — scale-up versus scale-out choices matter more in communication-bound regimes ¹⁸. Cluster-level profiling across LLM workloads adds the hardware-generation dimension: newer GPUs (B200) can be faster yet less energy-efficient per FLOP than H100 for the same models, which shifts where power shaping pays ⁶⁶.

Checkpointing is the other discrete energy event. Its cost depends strongly on processor power states (C-states cut checkpoint energy up to ~13% and restart energy up to ~20% at no time cost) and on I/O configuration (async mounts consume ~25% less checkpoint energy than synchronous ones) ⁵⁵. Geo-distributed training adds a synchronisation-idle tax: PowerScale’s hierarchical sync-async aggregation, power-aware clustering, and adaptive sync frequency reduce energy up to ~4× while matching time-to-accuracy, mostly by not keeping GPUs idling at full power across continents ⁵². Online per-iteration energy optimisation frameworks for distributed ML workloads report ~16% mean energy savings for ~5% time increase without user intervention ⁷⁸. Together with Perseus’s straggler analysis ⁹, these results define the training-side toolkit: slow down what is not on the critical path, shape communication and checkpoint events, and do not pay full power for idle synchronisation.

Theme 7 — Algorithmic levers with sign-dependent payoffs: speculation and quantisation

Two widely promoted energy levers turn out to have payoffs whose sign depends on the workload. Speculative decoding’s energy case is indirect: it produces identical outputs faster, so at the same power draw, energy per token falls roughly with the speedup — out-of-the-box speedups of 2–3× imply similar energy-per-token ratios in the best case ⁴⁰. Direct benchmarking complicates the story: measured energy-savings factors range from 1.34× to 2.51× for strong drafters (EAGLE-2/3), but simpler drafters (CoGA/DyGA) consume more energy than vanilla decoding on several model–dataset pairs because their acceptance rates do not repay the draft compute ¹⁴. Coupling speculation with frequency control (running draft tokens at lower clocks) is where the lever becomes a power-shaping tool rather than a throughput one ⁸³⁶¹. Quantisation shows the same sign-dependence at the model-size axis: weight-only quantisation increases energy ~25–45% on small models (1–3B, where kernels are launch-bound and dequantisation overhead dominates) while saving ~23% on 6–9B models — and the cross-over is sharp enough that “quantisation saves energy” is not a safe default claim ⁸⁸. Edge studies similarly find accuracy-energy trade-offs that depend on quantisation level and hardware ²⁴. Decoding strategy itself is an energy axis: sampling parameters and search strategy change GPU energy materially even when output quality barely moves ⁵⁷.

Theme 8 — What the smoothing buys at the grid edge

The motivation for all of the above — and increasingly a design constraint — is the grid interface. Transmission-connected AI datacentres exhibit rapid ramping and oscillatory load behaviour that operators (e.g., Alberta’s AESO) explicitly flag; coordinated batch-size control plus battery storage cut the storage’s discharge energy by ~71% and peak discharge power by ~51% while meeting a 10 MW/min ramp limit ⁴⁴. Field evidence that GPU clusters can behave as grid-interactive assets is real but young: a 130 kW GPU cluster achieved 100% compliance across 200+ grid power targets, including 40% load reduction within ~1 minute and <40-second zero-notice ramp-downs ⁸⁰. On the hardware side, power-electronics buffering can absorb GPU workload transients locally — an active power buffer cut line-side peak draw up to 45% in a data-centre power-delivery testbed ¹⁹. And quantisation gains a second life at the grid: trading a few percent of model quality for the ability to shed load fast, quantisation-enabled demand response cut total datacentre operating cost ~34% in simulation without curtailing served tokens ¹². The grid-scale half of this story — demand response, carbon-aware scheduling, UPS/battery flexibility — is covered in depth by the site’s model-to-grid review; the point here is that the kernel- and scheduler-level levers above are what make those facility-level promises executable at second-to-minute timescales.

Where the evidence disagrees

Capping decode: inert or effective? The strongest disagreement is whether power capping helps LLM inference at all. One set of measurements finds caps essentially inert in decode — the phase dominating production serving — because memory-bound phases never approach the cap, and attributes apparent cap effects to firmware throttling confounds ⁴⁸. Others report clean energy savings from caps on inference workloads including decode-heavy serving ⁶⁵⁶⁸³². The reconciliation is probably workload and hardware specific: caps bind when the workload is compute-bound or when many requests co-schedule into compute pressure, and the H200 generation’s HBM power behaviour (cap violations at 200 W) makes cross-generation generalisation unsafe ⁷². The disagreement is real and unresolved in the literature, and it matters operationally: a fleet-wide cap policy tuned on compute benchmarks may be doing nothing — or worse, adding latency through throttle oscillation — on decode-dominated traffic.

How much does frequency reduction cost in latency? Reported latency costs for aggressive DVFS range from negligible (1–6% for a 42% energy cut ⁵⁰; ~20% for a ~50% decode energy cut ⁸⁶) to material (9–10% TTFT/TPOT overheads for the biggest savings ⁸⁴), to catastrophic when governors are misconfigured (23–40% on mobile defaults ⁸⁷). Part of the spread is phase: decode tolerates low clocks, prefill does not ⁴². Part is SLO metric: TTFT and TPOT respond differently to the same frequency policy ⁸⁶⁸⁴. Part is generation: memory-clock behaviour differs sharply between H100 and H200 ⁷². Claims like “frequency scaling is free” and “frequency scaling destroys latency” are both supported somewhere; the honest synthesis is that the trade-off is phase-, metric-, and hardware-specific and must be measured per deployment.

Speculation and quantisation as energy levers. Both are widely asserted to save energy; both have published counterexamples — drafters that cost more energy than vanilla decoding ¹⁴ and quantisation that increases energy on small models ⁸⁸. The disagreement is not about mechanism but about the boundary conditions, which the promotional literature routinely omits.

Does disaggregation save energy? Phase-disaggregated serving systems report large energy gains from phase-separated frequency control ⁵⁸⁶¹⁵, while a direct re-examination finds disaggregation’s stage-wise frequency freedom does not yield energy savings unless load creates idle capacity ⁴¹. The difference is likely in baselines, load regimes, and whether the comparison holds power constant; no head-to-head arbitration exists yet.

Measurement methodology itself is contested. Papers disagree on what to measure: MLPerf Power standardises energy across the stack ⁷¹; TokenPowerBench argues for phase-aligned, meter-free power benchmarking ⁵⁹; Watt Counts argues that GPU choice dominates everything and benchmarks must span heterogeneous hardware ³. And the ma2026illusion result warns that firmware clock behaviour can corrupt exactly the throughput measurements used to evaluate caps ⁴⁸. Until the field agrees on measuring power waveforms rather than only energy totals, many “energy savings” claims will remain partially confounded.

Gaps and open questions

Kernel-level power shaping is strikingly thin. For all the attention to kernels in the performance literature, only a handful of works shape power at kernel granularity: kernel-level DVFS for LLM training ⁶⁷, kernel-fusion-for-power from 2011 ⁷⁶, kernel-scheduling energy effects ⁷⁷, and operator-level frequency separation in AFlex ²². Modern kernel engineering (FlashAttention-class fusions) is optimised for speed and memory, and its power-waveform consequences are essentially unstudied — a gap that matters because fused kernels change both peak current and ramp shape. What would settle it: power-instrumented kernel benchmarks (waveform, not just energy) across the standard LLM kernel zoo.

No ripple metric exists. The field measures energy (J), throughput, and occasionally peak power, but there is no standard metric for ripple — power variance, ramp-rate distribution, or overshoot events — despite grid operators caring about exactly those quantities ⁴⁴⁴³. MLPerf Power measures efficiency, not smoothness ⁷¹. The “Smoothing the Ramp” paper had to build its own coincidence-factor model ⁴³. A standardised ripple metric (analogous to SPECpower’s scaling curves) would let the smoothing claims in this review be compared across systems.

Scale is emulated, not demonstrated. The marquee training result (Perseus) is validated at 1,024-GPU scale only in emulation ⁹; the largest real deployments are a 130 kW cluster ⁸⁰ and single-digit-GPU testbeds. Whether Perseus-style schedules, cluster capping, and ramp shaping compose at 10,000-GPU scale with real stragglers, checkpoints, and grid contracts is untested in the public record.

A widely cited title does not exist. The framing “Perseus for inference” circulates in talks and blog posts, but no distinct publication of that name exists: the verified record is “Reducing Energy Bloat in Large Model Training” (SOSP 2024, arXiv title “Perseus: …Training”) ⁹. Claims about an inference version should cite the phase-aware serving systems instead ⁸⁶⁵²².

Vendor feasibility is underspecified. Several techniques require capabilities that datacentre GPU drivers restrict: per-kernel clock changes ⁶⁷, SM clock locking ⁴⁸, and undervolting ³⁹ all sit behind vendor-controlled interfaces whose production behaviour (and licensing) is not documented in the academic record. The frequency-switching-latency measurements ⁷⁴ suggest the control plane can keep up; whether the vendor plane permits it is another question.

Undershoot/overshoot at the voltage regulator is a hardware gap. Power-electronics work shows buffering can absorb GPU transients ¹⁹, and silicon work shows voltage guardbands exist partly to absorb noise ³⁹, but no public study connects kernel-level workload shaping to VRM transient behaviour on datacentre GPUs — the di/dt question remains open.

Confidence and limitations

Confidence is moderate overall, with a clear gradient. The energy-sweet-spot results (Themes 2–3) are replicated across independent groups, hardware generations, and both training and inference ^8650696564 — those conclusions carry high confidence. The decode-insensitivity and cap-inertia results are newer but consistent across at least four independent measurements ^48504272. The ripple-smoothing and grid-interface results are the least mature: mostly 2025–2026 preprints, single groups, and emulation/simulation for the fleet-level claims ⁴³⁴⁴⁸⁰ — those conclusions are provisional.

Limitations of this review: 50 of 89 sources were abstract-only (paywalled IEEE/ACM/Springer items with no open full text), so claims from them are capped at what abstracts state; two selected sources were dropped for having no retrievable abstract anywhere; one IEEE TSTE paper on energy-storage smoothing of training fluctuations was dropped for the same reason; Semantic Scholar was rate-limited throughout, so recall there is thinner than intended; and the 2026-heavy date distribution means several central results have not yet survived peer review or replication by other groups. Cross-cutting, the evidence base is NVIDIA-centric — AMD (MI300X) appears in two studies ³⁶⁶⁷ and other accelerators barely at all.