This whole argument would be dead in the water if society had de-carbonised 20 years ago instead of now. This stinks of the personal responsibility fallacy of carbon emissions when the real answer is to do the boring job of making energy production cleaner and doing a better job at moving people around.
Well said. Moralizing energy consumption is inefficient and no way to run a market. It'd be better to pass regulation that internalized the externalities in the price of electricity so that it captured the societal costs of emissions.
This should be seen as the opportunity of a lifetime. "Invest in infrastructure to power this awesome new technology" is obviously a more compelling story than "Invest in all new infrastructure—the plan is to use it less"
1. Google rolled our AI summaries on all of their search queries, through some very tiny model
2. Given worldwide search volume, that model now represents more than 50% of all queries if you throw it on a big heap with "intentional" LLM usage
3. Google gets to claim "the median is now 33x lower!", as the median is now that tiny model giving summaries nobody asked for
It's very concerning that this marketing puff piece is being eaten up by HN of all places as evidenced by the other thread.
Google is basing this all of "median" because there's orders of magnitudes difference betwen strong models (what most people think of when you talk AI) and tiny models, which Google uses "most" by virtue of running them for every single google search to produce the summaries. So the "median" will be whatever tiny model they use for those models. Never mind that Gemini 2.5 Pro, which is what everyone here would actually be using, may well consume >100x much.
It's absurdly misleading and rather obvious, but it feels like most are very eager to latch on to this so they can tell themselves their usage and work (for the many here in AI or at Google) is all peachy. I've been reading this place for years and have never before seen such uncritical adoption of an obvious PR piece detached from reality.
> make shit up on HN and be the top comment as long as it's to bash Google.
I don’t think that’s fair. Same would’ve happened if it were Microsoft, or Apple, or Amazon. By now we’re all used to (and tired) of these tech giants lying to us and being generally shitty. Additionally, for decades we haven’t been able to trust reports from big companies which say “everything is fine, really” when they publish it themselves, about themselves, contradicting the general wisdom of something bad they’ve been doing. Put those together and you have the perfect combination; we’re primed to believe they’re trying to deceive us again, because that’s what happens most of the time. It has nothing to do with it being Google, they just happened to be the target this time.
>It's very concerning that you can just make shit up on HN and be the top comment as long as it's to bash Google.
Off topic. I wanted to say somewhat counterintuitively I often upvote / submit things I disagree with and dont downvote it as long as sub comments offer a good counter argument or explanation.
Sometimes being top just meant that is what most people are thinking, and it being wrong and corrected is precisely why I upvote it and wish it stayed on top so others can learn.
As others have pointed out, this is false.
Google has made their models and hardware more efficient, you can read the linked report. Most of the efficiency comes from quantization, MoE, new attention techniques, and distillation (making smaller models useable in place of bigger models)
- The report doesn't name any Gemini models at all, only competitors'. Wonder why that is? If the models got so much more efficient, they'd be eager to show this.
- The report doesn't name any averages (means), only medians. Why oh why would they be doing this, when all other marketing pieces always use the average because outside of HN 99% of Joes on the street have no idea what a median is/how it differs from the mean? The average is much more relevant here when "measuring the environmental impact of AI inference".
- The report doesn't define what any of the terms "Gemini Apps", "the Gemini AI assistant" or "Gemini Apps text prompt" concretely mean
Are you sure? It wouldn’t shock me, but they specifically say “Gemini Apps”. I wasn’t familiar with the term, but a web search indicated that it has a specific meaning, and it doesn’t seem to me like web search AI summaries would be covered by it. Am I missing something?
I know there's a lot of rebuttals to this statement already, but I think there's a simpler way of showing it is incorrect:
Figure 2 in the paper shows the LMArena score of whatever model is used for "median" Gemini query. That score is consistent with Gemini Flash (probably 2.0, given the numbers are from May), not a "tiny model" used for summaries nobody is asking for.
I’ve been covering tech for 20 years. No, it wasn’t always like that.
There was a sincere mutual respect between the companies and the media industry that I don’t see anymore.
Both sides have their fault, but you know it’s not media that huperscaled and created gazillionaires by the score.
Also, software is way more bendable to the emperors’ whims, and Google has become particularly hypocritical in the way it publicly represent itself.
What exactly are you basing this assertion on (other than your feelings)? Are you accusing Google of lying when they say in the technical report [1]:
> This impact results from: A 33x reduction in per-prompt energy consumption driven by software efficiencies—including a 23x reduction from model improvements, and a 1.4x reduction from improved machine utilization.
followed by a list of specific improvements they've made?
Unless marketing blogs from any company specifically say what model they are talking about, we should always assume they're hiding/conflating/mislabeling/misleading in every way possible. This is corporate media literacy 101.
The burden of proof is on Google here. If they've reduced gemini 2.5 energy use by 33x, they need to state that clearly. Otherwise a we should assume they're fudging the numbers, for example:
A) they've chosen one particular tiny model for this number
or
B) it's a median across all models including the tiny one they use for all search queries
EDIT: I've read over the report and it's B) as far as I can see
Without more info, any other reading of this is a failing on the reader's part, or wishful thinking if they want to feel good about their AI usage.
We should also be ready to change these assumptions if Google or another reputable party does confirm this applies to large models like Gemini 2.5, but should assume the least impressive possible reading until that missing info arrives.
Even more useful info would be how much electricity Google uses per month, and whether that has gone down or continued to grow in the period following this announcement. Because total energy use across their whole AI product range, including training, is the only number that really matters.
If you have a market for it, the hardware industry will aggressively dig in to try to deliver. Maximum performance and maximum efficiency. So I can imagine there is still more to go.
I'm sure the relatively clean directed computational graph + massively parallel + massively hungry workload of AI is a breath of fresh air to the industry.
Hardware gains were for the longest time doing very little for consumers because the bottlenecks were not in the hardware but instead in extremely poorly written software running in very poorly designed layers of abstraction that nothing could be done about.
The hardware overhang embodied: that early AI will be inefficiently embodied as a blob of differentiable floating point numbers in order to do gradient descent on them, and shortly after be translated into a dramatically simpler and faster form. An AGI that requires a full rack of H100s to run, suddenly appearing on single video game consoles. https://www.lesswrong.com/w/computing-overhang
Fun fact: Deep Blue was a dedicated chess compute cluster that ran on 30 RS/6000 processors and 480 VLSI chips. If the Stockfish chess program existed in 1997 it would have beaten it with a single 486 CPU: https://www.lesswrong.com/posts/75dnjiD8kv2khe9eQ/measuring-...
There are two ways to make AI cheaper: make energy cheaper or make AI hardware and algorithms more efficient and use less energy that way. Google is investing in doing both. And that's a good thing.
I actually see growth in energy demand because of AI or other reasons as a positive thing. It's putting pressure on the world to deliver more energy cheaply. And it seems the most popular and straightforward way is through renewables + batteries. The more clean and cheap capacity like that is added, the more marginalized traditional more expensive solutions get.
The framing on this topic can be a bit political. I prefer to look at this through the lens of economics. The simple economic reality is that coal and gas plant construction has been bottle necked for years on a lot of things to the point where only very little of it gets planned and realized. And what little comes online has pretty poor economics. The cost and growth curves for renewables+battery paint a pretty optimistic picture here with traditional generation plateauing for a while (we'll still build more coal/gas plants, not a lot, and they'll be underutilized) and then dropping rapidly second half of the century as cost and availability of alternatives improves and completely steam roll anything that can't keep up. Fossil fuel based generation could be all but gone by the 2060s.
There are lots of issues with regulations, planning, approval, etc for fossil fuel based generation. There are issues with supply chains for things like turbines. Long term access to cooling water (e.g. rivers) is becoming problematic because of climate change. And there are issues with investors voting with their feet and being reluctant to make long term commitments in what could end up being very poor long term investments. A lot of this also impacts nuclear, which while clean remains expensive and hard to deliver. The net result of all this is that investments in new energy capacity are heavily biased towards battery + renewables. It's the only thing that works on short notice. And it's also the cheapest way to add new capacity. Current growth is already 80-90% renewable. It's not even close at this point. We're talking tens/hundreds of GW added annually.
Of course AI is so hungry for energy that there is a temporary increase in usage for coal/gas. That's existing underutilized plants temporarily getting utilized a bit more mainly because they are there and utilizing them a bit more is relatively easy and quick to realize. It's not actually cheaper and future cost reductions will likely come in the form of replacing that capacity with cheaper power generation as soon as that can be delivered.
There is a third way of making AI cheaper: using it less.
We have seen many technologies which have been made so much more efficient (heat pumps, solar panels, etc). Really great achievements. Yet the amount of (fossil) energy we use still grows.
Using less is always an individual choice. But not a realistic one to expect 8 billion+ people to take. That's also why fossil fuel usage is still increasing.
However, you might be too pessimistic here. Fossil fuel usage is actually widely expected to peak in the next few years and then enter a steady decline.
He uses a simple model with some very basic assumptions (conservative ones) where he shows how short term fossil fuel usage still increases. Mostly this is just market inertia. But then it will start decreasing and then some decades later, it declines all the way to zero with some long tail of hard to shift use cases.
He uses some very basic assumptions about economic growth continuing to grow by an average of 3%, a base assumption of renewables outgrowing energy demand increases by 3%, etc. You get to a modest fossil fuel decline by 2040, majority renewables powered economy by the 2050s. And virtually no fossil fuel left in the economy by 2065. The years change but the outcome stays the same as long as renewables outgrow demand increase.
There are lots of buts and ifs here but he's explicitly addressing the kind of pessimism you are voicing here.
Yes, and I think this is often a problem on the power generation side. The damn cost of energy storage and photovoltaics has been falling rapidly, hydropower and wind power costs are not high at all.
The key is to take advantage of economies of scale: The cost of renewable energy generation is mainly in the initial investment and equipment.
As long as you mass-produce enough equipment, the cost of each device will decrease due to economies of scale. However, thermal power generation is different. The cost of thermal power generation is mainly fuel, and the lower limit of the cost is much higher than photovoltaic power generation.
I don’t understand why so many people are obsessed with using fossil fuels for power generation, as if it is really more efficient... thermal power no longer has a price advantage a few years ago.
The average person doesn't care enough about not using fossil fuels to lower his quality of life. If your plan of action is moralizing at them until they do we might as well nuke ourselves back into the stone age for all the effect it will have.
The benefits of technical solutions is that you get the desired effect without any real trade-offs. I don't really care if I use a boiler or a heat pump to heat my house, because the end goal is to heat my house. I don't really care if I use an electric car or dead dinosaurs car, I just want to get places.
Make the efficient, more climate-friendly alternative a better deal and most people will switch. Tell people that they should give up their cars and AC because the planet will be 3C warmer in 100 years and you'll get an eye-roll. If you want the more environmentally-friendly but also more expensive option to win then the only real option is government subsidies, not preaching - enlightened self-interest trumps all.
Measurements for water consumption seems cherry-picked and incorrect to look better than they actually are. When asked about it, they doubled-down and incorrectly mentioned the study in question (to which they compared against) was incorrect. See https://www.linkedin.com/posts/shaolei-ren-68557415_today-go...
If it’s like Marvel sequels every year then there is a significant added training cost as the expectations get higher and higher to churn out better models every year like clockwork.
Cost/prompt is a ratio. "Prompt" is not a normalized metric that is stable over time. It can increase (as context lengths increase) or decrease (as google's product suite integrates llms).
Readit News
Playing whack a mole with individual behavior while the elephant in the room is energy production and transportation remains asinine as always.
1. Google rolled our AI summaries on all of their search queries, through some very tiny model 2. Given worldwide search volume, that model now represents more than 50% of all queries if you throw it on a big heap with "intentional" LLM usage 3. Google gets to claim "the median is now 33x lower!", as the median is now that tiny model giving summaries nobody asked for
It's very concerning that this marketing puff piece is being eaten up by HN of all places as evidenced by the other thread.
Google is basing this all of "median" because there's orders of magnitudes difference betwen strong models (what most people think of when you talk AI) and tiny models, which Google uses "most" by virtue of running them for every single google search to produce the summaries. So the "median" will be whatever tiny model they use for those models. Never mind that Gemini 2.5 Pro, which is what everyone here would actually be using, may well consume >100x much.
It's absurdly misleading and rather obvious, but it feels like most are very eager to latch on to this so they can tell themselves their usage and work (for the many here in AI or at Google) is all peachy. I've been reading this place for years and have never before seen such uncritical adoption of an obvious PR piece detached from reality.
> It's very concerning that this marketing puff piece is being eaten up by HN of all places as evidenced by the other thread.
It's very concerning that you can just make shit up on HN and be the top comment as long as it's to bash Google.
> Never mind that Gemini 2.5 Pro, which is what everyone here would actually be using, may well consume >100x much
Yes, exactly, never mind that. The report is to compare against a data point from May 2024, before Gemini 2.5 Pro became a thing.
Deleted Comment
I don’t think that’s fair. Same would’ve happened if it were Microsoft, or Apple, or Amazon. By now we’re all used to (and tired) of these tech giants lying to us and being generally shitty. Additionally, for decades we haven’t been able to trust reports from big companies which say “everything is fine, really” when they publish it themselves, about themselves, contradicting the general wisdom of something bad they’ve been doing. Put those together and you have the perfect combination; we’re primed to believe they’re trying to deceive us again, because that’s what happens most of the time. It has nothing to do with it being Google, they just happened to be the target this time.
Off topic. I wanted to say somewhat counterintuitively I often upvote / submit things I disagree with and dont downvote it as long as sub comments offer a good counter argument or explanation.
Sometimes being top just meant that is what most people are thinking, and it being wrong and corrected is precisely why I upvote it and wish it stayed on top so others can learn.
- The report doesn't name any averages (means), only medians. Why oh why would they be doing this, when all other marketing pieces always use the average because outside of HN 99% of Joes on the street have no idea what a median is/how it differs from the mean? The average is much more relevant here when "measuring the environmental impact of AI inference".
- The report doesn't define what any of the terms "Gemini Apps", "the Gemini AI assistant" or "Gemini Apps text prompt" concretely mean
It's very concerning that you claim this without previously fully reading and understanding Google's publication...
Figure 2 in the paper shows the LMArena score of whatever model is used for "median" Gemini query. That score is consistent with Gemini Flash (probably 2.0, given the numbers are from May), not a "tiny model" used for summaries nobody is asking for.
But, wasn't it always so?
Wasn't it always so in business of all kinds?
Why should we expect anything different? We should have been skeptical all along.
Deleted Comment
Deleted Comment
> This impact results from: A 33x reduction in per-prompt energy consumption driven by software efficiencies—including a 23x reduction from model improvements, and a 1.4x reduction from improved machine utilization.
followed by a list of specific improvements they've made?
[1] https://services.google.com/fh/files/misc/measuring_the_envi...
The burden of proof is on Google here. If they've reduced gemini 2.5 energy use by 33x, they need to state that clearly. Otherwise a we should assume they're fudging the numbers, for example:
A) they've chosen one particular tiny model for this number
or
B) it's a median across all models including the tiny one they use for all search queries
EDIT: I've read over the report and it's B) as far as I can see
Without more info, any other reading of this is a failing on the reader's part, or wishful thinking if they want to feel good about their AI usage.
We should also be ready to change these assumptions if Google or another reputable party does confirm this applies to large models like Gemini 2.5, but should assume the least impressive possible reading until that missing info arrives.
Even more useful info would be how much electricity Google uses per month, and whether that has gone down or continued to grow in the period following this announcement. Because total energy use across their whole AI product range, including training, is the only number that really matters.
I'm sure the relatively clean directed computational graph + massively parallel + massively hungry workload of AI is a breath of fresh air to the industry.
Hardware gains were for the longest time doing very little for consumers because the bottlenecks were not in the hardware but instead in extremely poorly written software running in very poorly designed layers of abstraction that nothing could be done about.
Fun fact: Deep Blue was a dedicated chess compute cluster that ran on 30 RS/6000 processors and 480 VLSI chips. If the Stockfish chess program existed in 1997 it would have beaten it with a single 486 CPU: https://www.lesswrong.com/posts/75dnjiD8kv2khe9eQ/measuring-...
I actually see growth in energy demand because of AI or other reasons as a positive thing. It's putting pressure on the world to deliver more energy cheaply. And it seems the most popular and straightforward way is through renewables + batteries. The more clean and cheap capacity like that is added, the more marginalized traditional more expensive solutions get.
The framing on this topic can be a bit political. I prefer to look at this through the lens of economics. The simple economic reality is that coal and gas plant construction has been bottle necked for years on a lot of things to the point where only very little of it gets planned and realized. And what little comes online has pretty poor economics. The cost and growth curves for renewables+battery paint a pretty optimistic picture here with traditional generation plateauing for a while (we'll still build more coal/gas plants, not a lot, and they'll be underutilized) and then dropping rapidly second half of the century as cost and availability of alternatives improves and completely steam roll anything that can't keep up. Fossil fuel based generation could be all but gone by the 2060s.
There are lots of issues with regulations, planning, approval, etc for fossil fuel based generation. There are issues with supply chains for things like turbines. Long term access to cooling water (e.g. rivers) is becoming problematic because of climate change. And there are issues with investors voting with their feet and being reluctant to make long term commitments in what could end up being very poor long term investments. A lot of this also impacts nuclear, which while clean remains expensive and hard to deliver. The net result of all this is that investments in new energy capacity are heavily biased towards battery + renewables. It's the only thing that works on short notice. And it's also the cheapest way to add new capacity. Current growth is already 80-90% renewable. It's not even close at this point. We're talking tens/hundreds of GW added annually.
Of course AI is so hungry for energy that there is a temporary increase in usage for coal/gas. That's existing underutilized plants temporarily getting utilized a bit more mainly because they are there and utilizing them a bit more is relatively easy and quick to realize. It's not actually cheaper and future cost reductions will likely come in the form of replacing that capacity with cheaper power generation as soon as that can be delivered.
We have seen many technologies which have been made so much more efficient (heat pumps, solar panels, etc). Really great achievements. Yet the amount of (fossil) energy we use still grows.
However, you might be too pessimistic here. Fossil fuel usage is actually widely expected to peak in the next few years and then enter a steady decline.
Michael Liebreich of Bloomberg NEF did a pretty interesting editorial on this decline a few weeks ago: https://about.bnef.com/insights/clean-energy/liebreich-the-p...
He uses a simple model with some very basic assumptions (conservative ones) where he shows how short term fossil fuel usage still increases. Mostly this is just market inertia. But then it will start decreasing and then some decades later, it declines all the way to zero with some long tail of hard to shift use cases.
He uses some very basic assumptions about economic growth continuing to grow by an average of 3%, a base assumption of renewables outgrowing energy demand increases by 3%, etc. You get to a modest fossil fuel decline by 2040, majority renewables powered economy by the 2050s. And virtually no fossil fuel left in the economy by 2065. The years change but the outcome stays the same as long as renewables outgrow demand increase.
There are lots of buts and ifs here but he's explicitly addressing the kind of pessimism you are voicing here.
The key is to take advantage of economies of scale: The cost of renewable energy generation is mainly in the initial investment and equipment.
As long as you mass-produce enough equipment, the cost of each device will decrease due to economies of scale. However, thermal power generation is different. The cost of thermal power generation is mainly fuel, and the lower limit of the cost is much higher than photovoltaic power generation.
I don’t understand why so many people are obsessed with using fossil fuels for power generation, as if it is really more efficient... thermal power no longer has a price advantage a few years ago.
Then you have no solution at all.
The benefits of technical solutions is that you get the desired effect without any real trade-offs. I don't really care if I use a boiler or a heat pump to heat my house, because the end goal is to heat my house. I don't really care if I use an electric car or dead dinosaurs car, I just want to get places.
Make the efficient, more climate-friendly alternative a better deal and most people will switch. Tell people that they should give up their cars and AC because the planet will be 3C warmer in 100 years and you'll get an eye-roll. If you want the more environmentally-friendly but also more expensive option to win then the only real option is government subsidies, not preaching - enlightened self-interest trumps all.
If it’s like Marvel sequels every year then there is a significant added training cost as the expectations get higher and higher to churn out better models every year like clockwork.