Readit News logoReadit News
overgard commented on Are OpenAI and Anthropic Losing Money on Inference?   martinalderson.com/posts/... · Posted by u/martinald
jsnell · 4 hours ago
For the top few providers, the training is getting amortized over absurd amount of inference. E.g. Google recently mentioned that they processed 980T tokens over all surfaces in June 2025.

The leaked OpenAI financial projections for 2024 showed about equal amount of money spent on training and inference.

Amortizing the training per-query really doesn't meaningfully change the unit economics.

> Fact remains when all costs are considered these companies are losing money and so long as the lifespan of a model is limited it’s going to stay ugly. Using that apartment building analogy it’s like having to knock down and rebuild the building every 6 months to stay relevant. That’s simply not a viable business model.

To the extent they're losing money, it's because they're giving free service with no monetizaton to a billion users. But since the unit costs are so low, monetizing those free users with ads will be very lucrative the moment they decide to do so.

overgard · 20 minutes ago
Assuming users accept those ads. Like, would they make it clear with a "sponsored section", or would they just try to worm it into the output? I could see a lot of potential ways that users reject the ad service, especially if it's seen to compromise the utility or correctness of the output.
overgard commented on Are OpenAI and Anthropic Losing Money on Inference?   martinalderson.com/posts/... · Posted by u/martinald
Aurornis · 4 hours ago
Exactly. All of the claims that OpenAI is losing money on every request are wrong. OpenAI hasn’t even unlocked all of their possible revenue opportunities from the free tier such as ads (like Google search), affiliate links, and other services.

There’s also a lot of comments in this thread who want LLM companies to fail for different reasons, so they’re projecting that wish on to imagined unit economics.

I’m having flashbacks to all of the conversations about Uber and claims that it was going to collapse as soon as the investment money ran out. Then Uber gradually transitioned to profitability and the critics moved to using the same shtick on AI companies.

overgard · 27 minutes ago
If they're profitable, why on earth are they seeking crazy amounts of investment month after month? It seems like they'll raise 10 billion one month, and then immediately turn around and raise another 10 billion a month or two after that. If it's for training, it seems like a waste of money since GPT-5 doesn't seem like it's that much of an improvement.
overgard commented on Are OpenAI and Anthropic Losing Money on Inference?   martinalderson.com/posts/... · Posted by u/martinald
dcre · an hour ago
What are we meant to take away from the 8000 word Zitron post?

In any case, here is what Anthropic CEO Dario Amodei said about DeepSeek:

"DeepSeek produced a model close to the performance of US models 7-10 months older, for a good deal less cost (but not anywhere near the ratios people have suggested)"

"DeepSeek-V3 is not a unique breakthrough or something that fundamentally changes the economics of LLM’s; it’s an expected point on an ongoing cost reduction curve. What’s different this time is that the company that was first to demonstrate the expected cost reductions was Chinese."

https://www.darioamodei.com/post/on-deepseek-and-export-cont...

We certainly don't have to take his word for it, but the claim is that DeepSeek's models are not much more efficient to train or inference than closed models of comparable quality. Furthermore, both Amodei and Sam Altman have recently claimed that inference is profitable:

Amodei: "If you consider each model to be a company, the model that was trained in 2023 was profitable. You paid $100 million, and then it made $200 million of revenue. There's some cost to inference with the model, but let's just assume, in this cartoonish cartoon example, that even if you add those two up, you're kind of in a good state. So, if every model was a company, the model, in this example, is actually profitable.

What's going on is that at the same time as you're reaping the benefits from one company, you're founding another company that's much more expensive and requires much more upfront R&D investment. And so the way that it's going to shake out is this will keep going up until the numbers go very large and the models can't get larger, and then it'll be a large, very profitable business, or, at some point, the models will stop getting better, right? The march to AGI will be halted for some reason, and then perhaps it'll be some overhang. So, there'll be a one-time, 'Oh man, we spent a lot of money and we didn't get anything for it.' And then the business returns to whatever scale it was at."

https://cheekypint.substack.com/p/a-cheeky-pint-with-anthrop...

Altman: "If we didn’t pay for training, we’d be a very profitable company."

https://www.theverge.com/command-line-newsletter/759897/sam-...

overgard · 33 minutes ago
In terms of sources, I would trust Zitron a lot more than Altman or Amodei. To be charitable, those CEOs are known for their hyperbole and for saying whatever is convenient in the moment, but they certainly aren't that careful about being precise or leaving out inconvenient details. Which is what a CEO should do, more or less, but, I wouldn't trust their word on most things.
overgard commented on Are OpenAI and Anthropic Losing Money on Inference?   martinalderson.com/posts/... · Posted by u/martinald
thatguysaguy · 2 hours ago
Why would you think that deepseek is more efficient than gpt-5/Claude 4 though? There's been enough time to integrate the lessons from deepseek.
overgard · 37 minutes ago
Because to make GPT-5 or Claude better than previous models, you need to do more reasoning which burns a lot more tokens. So, your per-token costs may drop, but you may also need a lot more tokens.
overgard commented on Are OpenAI and Anthropic Losing Money on Inference?   martinalderson.com/posts/... · Posted by u/martinald
overgard · 41 minutes ago
I thought the thing that made DeepSeek interesting (besides competition from China) was that its inference costs were something like 1/10th. So unless that gap has been bridged (has it?) I don't think a calculation based on DeepSeek can apply to OpenAI or Anthropic.
overgard commented on Claude for Chrome   anthropic.com/news/claude... · Posted by u/davidbarker
overgard · 2 days ago
From a privacy and security standpoint, hell no!
overgard commented on How to build a coding agent   ghuntley.com/agent/... · Posted by u/ghuntley
overgard · 4 days ago
I'd love for this kind of thing without the incredibly obnoxious commentary. I felt like I was reading propaganda rather than a how-to.
overgard commented on Copilot broke audit logs, but Microsoft won't tell customers   pistachioapp.com/blog/cop... · Posted by u/Sayrus
overgard · 9 days ago
I don’t know much about audit logs, but the more concerning thing to me is it sounds like it’s up to the program reading the file to register an access? Shouldn’t that be something at the file system level? I’m a bit baffled why this is a copilot bug instead of a file system bug unless copilot has special privileges? (Also to that: ick!)
overgard commented on Do things that don't scale, and then don't scale   derwiki.medium.com/do-thi... · Posted by u/derwiki
overgard · 12 days ago
Nitpick, but having side projects you don’t plan to scale isn’t anything unique to the AI era
overgard commented on Why LLMs can't really build software   zed.dev/blog/why-llms-can... · Posted by u/srid
byteknight · 14 days ago
I have to disagree. Anyone that says LLMs do not qualify as AI are the same people who will continue to move the goal posts for AGI. "Well it doesn't do this!". No one here is trying to replicate a human brain or condition in its entirety. They just want to replicate the thinking ability of one. LLMs represent the closest parallel we have experienced thus far to that goal. Saying that LLMs are not AI feel disingenuous at best and entirely purposely dishonest at the worst (perhaps perceived as staving off the impending demise of a profession).

The sooner people stop worrying about a label for what you feel fits LLMs best, the sooner they can find the things they (LLMs) absolutely excel at and improve their (the user's) workflows.

Stop fighting the future. Its not replacing right now. Later? Maybe. But right now the developers and users fully embracing it are experiencing productivity boosts unseen previously.

Language is what people use it as.

overgard · 14 days ago
The studies I've seen for AI actually improving productivity are a lot more modest than what the hype would have you believe. For example: https://www.youtube.com/watch?v=tbDDYKRFjhk

Skepticism isn't the same thing as fighting the future.

I will call something AGI when it can reliably solve novel problems it hasn't been pre-trained on. That's my goal post and I haven't moved it.

u/overgard

KarmaCake day10751November 16, 2009View Original