For years, data center efficiency came down to a few familiar numbers, most notably power usage effectiveness, which compares total facility energy to the energy used by IT equipment. Those metrics still matter, but AI is changing what efficiency means. As organizations build large AI factory sites, a new measure is getting more attention: how much useful AI output you get for each unit of energy and each dollar spent.
You may hear this described as tokens per watt, or even tokens per watt per dollar. Here is what the idea means, why it is catching on, and what operators can do to improve it.
What Is a Token?
In AI language models, a token is a small unit of text, often a word or part of a word. Models read and produce text one token at a time, so the number of tokens a system can process is a handy way to describe its output.
Tokens per watt asks a simple question: how many tokens can this infrastructure produce for each watt of power it uses? Adding cost to the equation brings in the economic side, asking how much that output costs.
Why Traditional Metrics Fall Short
Older efficiency measures focus on how well a facility delivers power to IT equipment. They do not say anything about whether that equipment is doing useful work.
- A facility can look efficient on paper while its GPUs sit partly idle.
- Two sites with identical efficiency ratings can produce very different amounts of AI output.
- Energy use alone does not capture the value created by a workload.
Output-based metrics tie energy directly to what the infrastructure produces, which is closer to how an AI business actually measures success.
What Drives Tokens per Watt
Several factors affect how much AI output a facility generates for each unit of energy.
- Hardware efficiency: Newer chips can often do more work per watt.
- Utilization: GPUs that stay busy deliver more value than those waiting for data or power.
- Power distribution: Fewer conversion steps mean less energy lost along the way.
- Cooling efficiency: Less energy spent on cooling leaves more for computing.
- Software and scheduling: Smart workload management keeps clusters productive.
- Reliability: Interruptions waste both time and energy.
The Role of Power and Cooling
Because AI hardware draws so much power, the infrastructure around it has a large effect on efficiency.
- Advanced power distribution: Higher-voltage designs and efficient equipment reduce losses.
- Liquid cooling: It can remove heat more efficiently than air at high densities, which may lower the energy needed for cooling.
- Modular, prefabricated designs: They can speed up deployment so capacity starts producing sooner.
- Monitoring and automation: Real-time data helps operators find waste and fix it.
Efficiency and Sustainability
Sustainability goals are part of the picture, too. Producing more AI output with fewer resources reduces operating costs and lowers environmental impact per unit of work. Many organizations now track both energy use and the productivity of that energy, rather than treating them as separate topics.
For broader context on how data center electricity demand is evolving, the International Energy Agency publishes widely cited analysis.
How Digital Tools Help
Digital twins and analytics software can help teams find and test efficiency gains before making changes in a live environment.
- Simulate cooling changes to see the effect on energy use.
- Model workload placement to avoid hot spots and wasted capacity.
- Track trends over time to spot gradual declines in performance.
- Predict maintenance needs so equipment stays reliable and efficient.
Practical Steps for Operators
- Measure what matters. Track useful output alongside energy and cost.
- Improve utilization first. Idle hardware is often the biggest source of waste.
- Review the whole power path. Look for conversion losses and opportunities to simplify.
- Match cooling to density. Use the right method for each part of the facility.
- Plan for growth. Efficiency gains are easier to build in than to retrofit.
- Review regularly. Targets and technologies change quickly.
Cautions When Using New Metrics
No single number tells the whole story. Tokens per watt can vary depending on the model, the type of workload, and how the measurement is taken. Make sure comparisons are fair, and use the metric as one of several tools rather than the only one.
A More Useful Question
Asking how much power a facility uses is a good start. Asking how much valuable work it produces for that power is a better one. As AI infrastructure grows, output-based measures like tokens per watt are likely to become a standard part of how operators plan, compare, and improve their facilities.
