The Carbon Cost of Thinking Machines

In April 2024, Google’s annual environmental report revealed something uncomfortable. After years of matching its energy consumption with renewable energy purchases and maintaining carbon-neutral operations since 2007, Google’s total greenhouse gas emissions had increased 48% over 2019 levels. The cause was not a failure of renewable energy procurement—Google was still buying enormous quantities of wind and solar power. The cause was that AI infrastructure was consuming energy faster than the company could source clean power to cover it.

That number—48% over five years, against a backdrop of corporate net-zero commitments—is worth sitting with. Google is one of the most sophisticated and well-resourced companies in the world when it comes to sustainability infrastructure. If they’re struggling, the rest of the industry is in a worse position.

The specific energy consumption of AI training and inference is difficult to quantify from outside the organizations doing it, because the major AI labs treat compute consumption as commercially sensitive. The numbers that are available come from academic papers, earnings call remarks, and occasional regulatory disclosures—and they should be treated as lower bounds, because the organizations with the worst numbers have the least incentive to disclose them.

Training a large language model requires running thousands of specialized chips (A100s, H100s, or their successors) in synchrony for weeks or months. The Llama 3 70B model, Meta reported, required approximately 6.4 million GPU-hours to train. An NVIDIA H100 GPU draws roughly 700 watts. Rough calculation: 6.4 million GPU-hours × 0.7 kWh per hour ≈ 4.5 million kWh, or about 4,500 MWh for that one training run.

That figure, by itself, seems modest. 4,500 MWh is approximately the annual electricity consumption of 400 American households. The problem is not any single training run—it’s the industrial scale at which training runs happen, combined with the much larger energy cost of inference at deployment scale.

Inference—running a model to answer a user query—uses far less energy per instance than training. But there are billions of queries. The IEA estimated in January 2024 that a ChatGPT query uses approximately 10 times more energy than a Google Search query. Google Search processes roughly 8.5 billion queries per day. If ChatGPT-scale AI assistants substitute for or add to this query volume at 10x the energy cost, the arithmetic becomes serious quickly.

Sam Altman acknowledged in early 2024 that OpenAI was facing a significant compute cost constraint, and that energy availability—not just compute availability—was becoming a limiting factor for scaling. Microsoft, as OpenAI’s primary infrastructure provider, announced a $100 billion investment in data center construction. Amazon Web Services announced similar commitments. Meta’s data center capital expenditure guidance for 2025 was $37-40 billion, a nearly 100% increase from 2024.

The geography of where this computing infrastructure gets built matters enormously for its carbon impact. Data centers built near abundant renewable energy—hydroelectric power in the Pacific Northwest, geothermal in Iceland, wind in west Texas—have materially lower emissions per unit of computation than data centers running on grid mixes heavy in natural gas or coal.

The current buildout is not location-agnostic. The constraint on data center expansion is not just land or fiber connectivity—it’s grid interconnection. Getting a new large data center connected to adequate power supply, especially with a commitment to clean power, involves multi-year queuing processes at grid operators and often requires negotiating directly with utilities or acquiring power purchase agreements from renewable developers.

Microsoft has been building in Virginia’s “Data Center Alley”—Prince William County and Loudoun County—partly because the grid interconnections are already there. Virginia’s grid mix is approximately 40% natural gas. Nuclear provides another 34%. Renewables are a minority. The physical infrastructure reality constrains the clean energy aspiration.

Google’s approach—and to varying degrees Amazon and Microsoft’s—is to match energy consumption with renewable energy certificates (RECs) and power purchase agreements on a geographic matching basis. This is better than buying generic RECs with no geographic or temporal alignment. It is not the same as running on real-time renewable power, and the distinction matters.

A data center drawing power from the Virginia grid at 2 AM on a cold winter night with low solar output and high heating demand is using coal and gas regardless of what power purchase agreements are on paper. The electrons on the grid don’t know about the contract; they follow Kirchhoff’s laws. The temporal and geographic matching issue is the central unresolved tension in tech sector clean energy claims.

Water is the less-discussed environmental cost. Modern AI chips generate substantial heat. Cooling that heat through evaporative cooling systems—the standard approach in large data centers—consumes significant water. Google’s 2023 environmental report acknowledged that its total operational water consumption was approximately 5.6 billion liters, up 20% from the prior year, largely due to AI infrastructure. Microsoft’s water use grew 34% in fiscal year 2023.

5.6 billion liters is roughly the annual water consumption of a city of 100,000 people. That’s not a negligible number, and in water-stressed regions—which include several major data center markets in the American Southwest—the trade-off between tech sector water use and other uses is not abstract.

The industry response to this pressure has been to pursue more aggressive liquid cooling approaches (direct liquid cooling of chips eliminates the evaporative cooling requirement) and to choose data center locations with access to sustainable water sources. The transition to liquid cooling is happening but slowly, because existing facilities were built for air and evaporative cooling and retrofitting them is expensive.

There’s a harder structural issue: the timing of the AI compute buildout. The current wave of data center construction is locking in physical infrastructure—buildings, power connections, cooling systems—that will operate for 15-25 years. The clean energy capacity to power these facilities at the required scale doesn’t exist today and will take years to build. The data centers, once constructed, will draw power from whatever is available on the local grid.

The investment community has priced AI infrastructure growth as unambiguously positive. The carbon math suggests a more complicated picture. The companies claiming net-zero operations while expanding compute capacity at 50-100% annually are either buying carbon offsets at scale (which have credibility problems) or are holding claims that don’t survive rigorous scrutiny.

Whether this makes AI a net contributor to climate change or a net reducer depends on a counterfactual question: what problems is AI solving, and how does the avoided impact compare to the direct emissions?

The genuine potential for AI to improve climate outcomes—through better crop management that reduces agricultural emissions, through grid optimization that enables higher renewable penetration, through climate modeling that improves the quality of adaptation planning, through materials discovery that accelerates battery or carbon capture technology—is real. Whether this potential gets realized, and at what speed, is deeply uncertain.

The direct emissions, by contrast, are already occurring and are measurable. A new data center being built in 2026 will operate for 20 years. The future AI applications that might justify its environmental cost are not yet deployed and may never materialize at the scale needed to close the accounting.

This is not an argument against AI infrastructure investment. It is an argument against the rhetorical move that says “AI will solve climate change” as a reason to wave away AI’s own environmental costs. The causal chain from data center construction to climate benefit runs through enormous uncertainties in deployment, adoption, policy, and physics. The causal chain from data center construction to grid demand runs through the laws of thermodynamics.

The GPU running a precision irrigation recommendation and the GPU running a celebrity deepfake draw the same power. The mix of applications actually deployed determines whether the investment was net-positive for the climate, and that mix is set by market demand, not by the optimistic scenarios in foundation model lab blog posts.

Google’s 48% emissions increase is the honest accounting breaking through the press release. The rest of the industry owes its stakeholders the same transparency.