This is all a familiar pattern. In the early days of ride share it was an amazing advancement and very cheap, because it was highly subsidized. The quality was decent, and certainly better than taxi and car services in most cities. Tons of ride share app companies popped up.
Then reality set in. Costs were raised so they weren’t losing money anymore. Rideshare became more of a commodity and competitors got squeezed out as there wasn’t much room to compete and make money. Service quality went downhill. Uber is generally reliable but the quality has fallen off a cliff. My last car smelled bad and the rear axel sounded like was about to fall off the car. In most cities at the airport I just walk outside and get an old fashioned taxi at the rank vs dealing with all the nonsense regulations forcing one to walking to some remote corner of a parking garage for the “ride share pickup” zone.
GenAI is entering that pivot point. The products have plateaued. There’s pressure to stop the loss leaders and set prices to a more realistic level. Services are becoming commoditized. It’s not going away but we’re entering a period of rapid consolidation. GenAI will still be here in a few years and will be useful, but like rideshare the allure will wear old and we’ll look at these things like we do spell checkers today. Something everyone uses but ultimately boring commoditized tech where there’s not a lot of money to be made. A useful feature to add to actual products.
I do think there’s some good opportunity to shift to locally run small models, but that too will just become commoditized spell-checker level tech.
I can't agree there's a plateau just a few weeks after two companies got gold medals at IOI and IMO using natural language (no Lean). Seems like progress is continuing nicely.
I am using the current models and they are still as useful as 6 or 12 months ago
The deal is still about the same: if you bother to do most of the hard part (thinking it through) the code generators can just about generate all the boilerplate
> In most cities at the airport I just walk outside and get an old fashioned taxi at the rank vs dealing with all the nonsense
Not the primary point of your post, but I am always evangelizing to my friends about this 'hack.' I can't believe that people are willing to walk half a mile and queue up in the rain/sun/snow to be driven by some random person who will probably make them listen to their demo tape, instead of just taking the myriad taxis that are sitting right there.
Takes probably 20-30 minutes off of my airport commute.
Is it fair to say that AI is continuously subsidized? Once a datacenter is built and becomes profitable, why would anyone just scrap it? Even if the owner goes into bankruptcy, that's a huge capital asset that evidently people can't get enough of.
It's kind of undeniable at this point that at least some parts of the AI boom have been really good for society. It just took a while to realize exactly where this was useful.
> In the early days of ride share it was an amazing advancement and very cheap, because it was highly subsidized.
This is not an analogous situation.
Inference APIs aren’t subsidised, and I’m not sure the monthly plans are any more either. AI startups burn a huge amount of money on providing free service to drive growth. That’s something they can reduce at any time without raising costs for their customers at all. Not to mention the fact that the cost of providing inference is plummeting by several orders of magnitude.
Uber weren’t providing free service to huge numbers of people, so when they wanted to turn a profit they couldn’t reduce there and had to raise prices for their customers. And the fees they pay to drivers didn’t drop a thousandfold so it wasn’t getting vastly cheaper to provide service.
The unit economics of these models and APIs are really ugly. Those saying they are not losing money on inference likely are only doing so when making up funky non-GAAP accounting thinking. It’s the old “we’re making money when you ignore all the places we’re spending money” argument.
When you factor in the R&D costs required to make these models and the very limited lifespan of a model (and thus extremely high capital investment depreciation rate) the numbers are pretty nasty.
This is hard to pin down. There are plenty of metal companies providing hosted inference at market rates (i.e. assumed profitably if heading towards some commodity price floor). The premise that every single one of these companies is operating at a loss is unlikely. The open question is about the "off-book" training costs for the models running on these servers: are your unit economics positive when factoring training costs. And if those training costs are truly off-book, it's not a meritless argument to say the model providers are "subsidizing" the inference industry. But it's not a clear cut argument either.
Anthropic and OpenAI are their own beasts. Are their unit economics negative? Depends on the time frame you're considering. In the mid-longer run, they're staking everything on "most decidedly not negative". But what are the rest of us paying on the day OpenAI posts 50% operating margins?
What makes you think these things aren’t subsidized? It would be very impressive if Claude was making money off of their $20/month users that hit their weekly limits.
How do you get to that conclusion? There is no inference without training, so each sale of a single inference token has a cost that includes both the inference as well as the amortised cost of training.
Sure they are - the big companies are dumping billions in capital on it, and the small companies are getting a firehose of venture, sovereign and pe to build stuff.
The way the big AI players are playing supports the assertion that the LLM is plateuing. The differentiator between OpenAI, Gemini, Copilot, Perpexity, Grok, etc is the app and how they find novel ways to do stuff. The old GPT models that Microsoft uses are kneecapped and suck, the Copilot for Office 365 is pretty awesome because it can integrate with the Office graph and has alot of context.
> AI startups burn a huge amount of money on providing free service to drive growth.
Of the pure-play companies, only OpenAI do this. Like, Anthropic are losing a bunch of money and the vast majority of their revenue comes from API usage.
So, either the training costs completely dominate the inference costs (seems unlikely but maybe) or they're just not great businesses.
I do think that OpenAI/Anthropic are probably hiring a lot of pre and post sales tech people to help customers use the products, and that's possibly something that they could cut in the future.
I think you may be looking past the point they are making. Rideshare was better, it was cheaper, it was nice. Its no longer better, cheaper, or nicer. They're doing fine for sure.... like the AI companies will be doing fine... but once the prices go up the ROI for AI agents won't be as appealing to every company. It may raise the bar higher for new companies/products rather than lower it.
I'm fairly certain these companies should pivot to selling / licensing AI "s/w drivers" for commodity consumer hardware that enables all these apps to run local-first or local-only.
The token cost stopped decaying as expected, as mentioned by the original 100,000k post on HN, and the move nowadays is towards more context to keep building functionality. The cost is just going to go up for inference. These companies might be better off splitting their focus between training and tooling, and canning all the capex/opex associated with inferrence.
Forget S/W engineers for a moment ... Every white collar worker I know, especially non technical folks, use ChatGPT all the time, and believe that is AI at this point. That demand isn't going to vanish overnight.
The counter argument is usually "They'll sell data", but I'm not sure you can double the number of trillion dollar data companies without some dilution of the market, and reach a billion devices / users without nation-state level infra.
It’s basically the old “what Intel giveth, Microsoft taketh away” but then with NVidia and AI shops.
Models get more computationally expensive as they start doing more things, and an equilibrium will be found what people are willing to pay per token.
I do expect the quality of output to increase incrementally, not exponentially, as models start using more compute. The real problem begins when companies like NVidia can’t make serious optimizations anymore, but history has proven that this seems unlikely.
The non S/W folks are currently all using it because it's free to a certain degree. There's no chance in hell they'll be paying for it. So the only other way for the AI companies to make money out of it is to add Ads to the whole shitshow, turning it into an even greater shitshow that not just dumbs down the planet but adds commercials on top of it.
> Because instead of writing code, they’re spending - wasting? - a ton of time fixing AI coding blunders. This is not a productive use of mid-level, never mind senior, programmers.
It’s amazing to think that humans have been writing blunder-free code all this time and only AI is making mistakes.
Humans coders, including good ones, make errors all the time and I don’t fully trust the code written by even my strongest team members (including myself; I’m far from the strongest programmer).
The claim is not that humans write fewer bugs than AI. The claim is that devoting senior time to fixing bugs in mass-produced AI code lowers overall quality.
This is true, but when good human developers introduce bugs, at least their code adheres to a thoughtful software design that matches expectations. My experience with AI code is its much less likely to meet that criteria.
I do think there will be a great de-skilling in software as we know it currently, essentially equivalent to the shift from craftsman manufacturing to assembly line production in the physical world. There will of course always be niches for high skill experts, but the vast majority of enterprise CRUD work will be done by people with far less expertise, whose salary will be some diminished percentage based on their reliance on AI. Will the results be "better"? No, but they will be good enough, and faster and more reliably tracked/scheduled and allow capital more control. The bosses have wanted to be able to "add X dollars to a project to speed it up by Y" forever, and now they finally can.
But will they be good enough? That’s a question that can only be answered after many years in a project’s lifecycle.
Not sure about allowing for more control, what happens when a complex/exotic issue that an AI is not able to solve arises? The bosses will have to pay a premium to whatever expert is left to address the problem.
>Not sure about allowing for more control, what happens when a complex/exotic issue that an AI is not able to solve arises?
What happens when a complex piece of machinery breaks down at the factory? You call in an expensive mechanic to fix it. You also still have engineers overseeing the day to day operations and processes to make sure that doesn't happen, but the bulk of the work is carried out by semi-skilled labor. It's not hard to imagine software going this way, as it's inevitably what capital wants.
Commented on this the other day, I think AI is fundamentally different from ride share apps and Uber for X services. It's more likely to follow a Moore's law trajectory. Getting cheaper and better over time?
Where as to get AI to any sort of approximation of what it's hyped up to be, may involve exponentially higher hardware costs.
So for the longest period of time, AI was sitting in about 90% accuracy. With the use of Nvidia hardware it's going to say 99 to 99.9%. I don't think it's actually 99.9%
To replace humans, I think you effectively need 99.999% and even more depending on the domain like self-driving is probably eight nines.
What's the hardware cost to get each one of those nines linear polynomial? Exponential?
Local and preferable Free. It seems odd that the majority of the software development world is gleefully becoming dependent upon proprietary tools running on someone else's machine.
This is untrue simply based on the so many past instances of Gemini, OpenAI making their products cheaper. The ratelimits for GPT 5 are pretty high. The API costs have decreased by 50% over and above o3's reduction which was also massive.
This is not even considering the fact that the performance has also increased.
While running locally will no doubt get cheaper over time (and hence become much more viable), cloud compute cost will also drop significantly as better hardware and more specialized models are created. We have seen this process already where the cost per million tokens has been falling rapidly.
It feels like a lot of the core LLM progress has plateaued or is approaching the end of the asymptote. We’re seeing a ton of great tooling being built around LLMs to increase their utility, but I wonder how much more juice we can really squeeze out of these things.
Readit News
Then reality set in. Costs were raised so they weren’t losing money anymore. Rideshare became more of a commodity and competitors got squeezed out as there wasn’t much room to compete and make money. Service quality went downhill. Uber is generally reliable but the quality has fallen off a cliff. My last car smelled bad and the rear axel sounded like was about to fall off the car. In most cities at the airport I just walk outside and get an old fashioned taxi at the rank vs dealing with all the nonsense regulations forcing one to walking to some remote corner of a parking garage for the “ride share pickup” zone.
GenAI is entering that pivot point. The products have plateaued. There’s pressure to stop the loss leaders and set prices to a more realistic level. Services are becoming commoditized. It’s not going away but we’re entering a period of rapid consolidation. GenAI will still be here in a few years and will be useful, but like rideshare the allure will wear old and we’ll look at these things like we do spell checkers today. Something everyone uses but ultimately boring commoditized tech where there’s not a lot of money to be made. A useful feature to add to actual products.
I do think there’s some good opportunity to shift to locally run small models, but that too will just become commoditized spell-checker level tech.
I am using the current models and they are still as useful as 6 or 12 months ago
The deal is still about the same: if you bother to do most of the hard part (thinking it through) the code generators can just about generate all the boilerplate
Yeah. Amazing.
Not the primary point of your post, but I am always evangelizing to my friends about this 'hack.' I can't believe that people are willing to walk half a mile and queue up in the rain/sun/snow to be driven by some random person who will probably make them listen to their demo tape, instead of just taking the myriad taxis that are sitting right there.
Takes probably 20-30 minutes off of my airport commute.
It's kind of undeniable at this point that at least some parts of the AI boom have been really good for society. It just took a while to realize exactly where this was useful.
This is not an analogous situation.
Inference APIs aren’t subsidised, and I’m not sure the monthly plans are any more either. AI startups burn a huge amount of money on providing free service to drive growth. That’s something they can reduce at any time without raising costs for their customers at all. Not to mention the fact that the cost of providing inference is plummeting by several orders of magnitude.
Uber weren’t providing free service to huge numbers of people, so when they wanted to turn a profit they couldn’t reduce there and had to raise prices for their customers. And the fees they pay to drivers didn’t drop a thousandfold so it wasn’t getting vastly cheaper to provide service.
When you factor in the R&D costs required to make these models and the very limited lifespan of a model (and thus extremely high capital investment depreciation rate) the numbers are pretty nasty.
> Inference APIs aren’t subsidised
This is hard to pin down. There are plenty of metal companies providing hosted inference at market rates (i.e. assumed profitably if heading towards some commodity price floor). The premise that every single one of these companies is operating at a loss is unlikely. The open question is about the "off-book" training costs for the models running on these servers: are your unit economics positive when factoring training costs. And if those training costs are truly off-book, it's not a meritless argument to say the model providers are "subsidizing" the inference industry. But it's not a clear cut argument either.
Anthropic and OpenAI are their own beasts. Are their unit economics negative? Depends on the time frame you're considering. In the mid-longer run, they're staking everything on "most decidedly not negative". But what are the rest of us paying on the day OpenAI posts 50% operating margins?
A lot of people disagreed with this point when I posted it, however Sam Altman said last week:
> We're profitable on inference. If we didn't pay for training, we'd be a very profitable company.
— https://www.axios.com/2025/08/15/sam-altman-gpt5-launch-chat...
How do you get to that conclusion? There is no inference without training, so each sale of a single inference token has a cost that includes both the inference as well as the amortised cost of training.
The way the big AI players are playing supports the assertion that the LLM is plateuing. The differentiator between OpenAI, Gemini, Copilot, Perpexity, Grok, etc is the app and how they find novel ways to do stuff. The old GPT models that Microsoft uses are kneecapped and suck, the Copilot for Office 365 is pretty awesome because it can integrate with the Office graph and has alot of context.
This made me laugh. Thanks for making my Friday a little bit better.
Of the pure-play companies, only OpenAI do this. Like, Anthropic are losing a bunch of money and the vast majority of their revenue comes from API usage.
So, either the training costs completely dominate the inference costs (seems unlikely but maybe) or they're just not great businesses.
I do think that OpenAI/Anthropic are probably hiring a lot of pre and post sales tech people to help customers use the products, and that's possibly something that they could cut in the future.
I may be wrong, but wasn’t compute part of Microsoft’s 2019 or 2023 investment deals with OpenAI?
The only thing which has gone downhill more is Airbnb.
At best a middling experience these days, and on average a poor experience.
The token cost stopped decaying as expected, as mentioned by the original 100,000k post on HN, and the move nowadays is towards more context to keep building functionality. The cost is just going to go up for inference. These companies might be better off splitting their focus between training and tooling, and canning all the capex/opex associated with inferrence.
Forget S/W engineers for a moment ... Every white collar worker I know, especially non technical folks, use ChatGPT all the time, and believe that is AI at this point. That demand isn't going to vanish overnight.
The counter argument is usually "They'll sell data", but I'm not sure you can double the number of trillion dollar data companies without some dilution of the market, and reach a billion devices / users without nation-state level infra.
Models get more computationally expensive as they start doing more things, and an equilibrium will be found what people are willing to pay per token.
I do expect the quality of output to increase incrementally, not exponentially, as models start using more compute. The real problem begins when companies like NVidia can’t make serious optimizations anymore, but history has proven that this seems unlikely.
It’s amazing to think that humans have been writing blunder-free code all this time and only AI is making mistakes.
Humans coders, including good ones, make errors all the time and I don’t fully trust the code written by even my strongest team members (including myself; I’m far from the strongest programmer).
At least they admit they have no idea what they are talking about.
Not sure about allowing for more control, what happens when a complex/exotic issue that an AI is not able to solve arises? The bosses will have to pay a premium to whatever expert is left to address the problem.
Deleted Comment
What happens when a complex piece of machinery breaks down at the factory? You call in an expensive mechanic to fix it. You also still have engineers overseeing the day to day operations and processes to make sure that doesn't happen, but the bulk of the work is carried out by semi-skilled labor. It's not hard to imagine software going this way, as it's inevitably what capital wants.
Why are people always speaking about CRUD? In 30 years, I haven't done anything related to CRUD. And I'm also very confused about it.
Or at least cheaper.
Right?
Where as to get AI to any sort of approximation of what it's hyped up to be, may involve exponentially higher hardware costs.
So for the longest period of time, AI was sitting in about 90% accuracy. With the use of Nvidia hardware it's going to say 99 to 99.9%. I don't think it's actually 99.9%
To replace humans, I think you effectively need 99.999% and even more depending on the domain like self-driving is probably eight nines.
What's the hardware cost to get each one of those nines linear polynomial? Exponential?
https://www.gnu.org/philosophy/who-does-that-server-really-s...
This is not even considering the fact that the performance has also increased.
Deleted Comment