Just because something exhibits an exponential growth at one point in time, that doesn’t mean that a particular subject is capable of sustaining exponential growth.
Their Covid example is a great counter argument to their point in that covid isn’t still growing exponentially.
Where the AI skeptics (or even just pragmatists, like myself) chime in is saying “yeah AI will improve. But LLMs are a limited technology that cannot fully bridge the gap between what they’re producing now, and what the “hypists” claim they’ll be able to do in the future.”
People like Sam Altman know ChatGPT is a million miles away from AGI. But their primary goal is to make money. So they have to convince VCs that their technology has a longer period of exponential growth than what it actually will have.
The argument is not that it will keep growing exponentially forever (obviously that is physically impossible), rather that:
- given a sustained history of growth along a very predictable trajectory, the highest likelihood short term scenario is continued growth along the same trajectory. Sample a random point on an s-curve and look slightly to the right, what’s the most common direction the curve continues?
- exponential progress is very hard to visualize and see, it may appear to hardly make any progress while far away from human capabilities, then move from just below to far above human very quickly
My point is that the limits of LLMs will be hit long before we they start to take on human capabilities.
The problem isn’t that exponential growth is hard to visualise. The problem is that LLMs, as advanced and useful a technique as it is, isn’t suited for AGI and thus will never get us even remotely to the stage of AGI.
The human like capabilities are really just smoke and mirrors.
It’s like when people anthropomorphisise their car; “she’s being temperamental today”. Except we know the car is not intelligence and it’s just a mechanical problem. Whereas it’s in the AI tech firms best interest to upsell the human-like characteristics of LLMs because that’s how they get VC money. And as we know, building and running models isn’t cheap.
The most common part of the S-curve by far is the flat bit before and the flat bit after. We just don't graph it because it's boring. Besides which there is no reason at all to assume that this process will follow that shape. Seems like guesswork backed up by hand waving.
So it's an argument impossible to counter because it's based on a hypothesis that is impossible to falsify: it predicts that there will either be a bit of progress, or a lot of progress, soon. Well, duh.
That feels like you're moving the goal posts a bit.
Exponential growth over the short term is very uninteresting. Exponential growth is exciting when it can compound.
E.g. if i offered you an investing opportunity 500% / per year compounded daily - that's amazing. If the fine print is that that rate will only last for the very near term (say a week), then it would be worse than a savings account.
> Just because something exhibits an exponential growth at one point in time, that doesn’t mean that a particular subject is capable of sustaining exponential growth.
Which is pretty ironic given the title of the post
>People notice that while AI can now write programs, design websites, etc, it still often makes mistakes or goes in a wrong direction, and then they somehow jump to the conclusion that AI will never be able to do these tasks at human levels, or will only have a minor impact. When just a few years ago, having AI do these things was complete science fiction!
Both things can be true, since they're orthogonal.
Having AI do these things was complete fiction 10 years ago. And after 5 years of LLM AI, people do start to see serious limits and stunted growth with the current LLM approaches, while also seeing that nobody has proposed another serious contended to that approach.
Similarly, going to the moon was science finction 100 years ago. And yet, we're now not only not in Mars, but 50+ years without a new moon manned landing. Same for airplanes. Science fiction in 1900. Mostly stale innovation wise for the last 30 years.
A lot of curves can fit an exponential line plot, without the progress going forward being exponential.
We would have 1 trillion transistor cpus following Moore's "exponential curve"
I agree with all your points, just wanted to say that transistor count is probably a counter example. We have been keeping with the Moore's Law more or less[1] and M3 Max, a 2023 consumer-grade CPU, has ~100B of transistors, "just" one order of magnitude away from yout 1T. I think that shows we haven't stagnated much in transistor density and the progress is just staggering!
That one order of magnitude is about 7 years behind the Moore's Law. We're still progressing but it's slower, more expensive and we hit way more walls than before.
Except it’s not been five years, it’s been at most three, since approximately no one was using LLMs prior to ChatGPT’s release, which was just under three years ago. We did have Copilot a year before that, but it was quite rudimentary.
And really, we’ve had even less than that. The first large scale reasoning model was o1, which was released 12 months ago. More useful coding agents are even newer than that. This narrative that we’ve been using these tools for many years and are now hitting a wall doesn’t match my experience at all. AI-assisted coding is way better than it was a year ago, let alone five.
>Except it’s not been five years, it’s been at most three,
Why would it be "at most" 3? We had Chat GPT commercially available as private beta API on 2020. It's only the mass public that got 3.5 3 years ago.
But those who'd do the noticing as per my argument is not just Joe Public (which could be oblivious), but people already starting in 2020, and includes people working in the space, who worked with LLM and LLM-like architectures 2-3 years before 2020.
> Given consistent trends of exponential performance improvements over many years and across many industries, it would be extremely surprising if these improvements suddenly stopped.
I'm sure people were saying that about commercial airline speeds in the 1970's too.
But a lot of technologies turn out to be S-shaped, not purely exponential, because there are limiting factors.
With LLM's at the moment, the limiting factors might turn out to be training data, cost, or inherent limits of the transformer approach and the fact that LLM's fundamentally cannot learn outside of their context window. Or a combination of all of these.
The tricky thing about S curves is, you never know where you are on them until the slowdown actually happens. Are we still only in the beginning of the growth part? Or the middle where improvement is linear rather than exponential? And then the growth starts slowing...
> a lot of technologies turn out to be S-shaped, not purely exponential, because there are limiting factors.
Yes of course it’s not going to increase exponentially forever.
The point is, why predict that the growth rate is going to slow exactly now? What evidence are you going to look at?
It’s possible to make informed predictions (eg “Moore’s law can’t get you further than 1nm with silicon due to fundamental physical limits”). But most commenters aren’t basing their predictions in anything as rigorous as that.
And note, there are good reasons to predict a speedup, too; as models get more intelligent, they will be able to accelerate the R&D process. So quality per-researcher is now proportional to the exponential intelligence curve, AND quantity of researchers scales with number of GPUs (rather than population growth which is much slower).
NOTE IN ADVANCE: I'm generalizing, naturally, because talking about specifics would require an essay and I'm trying to write a comment.
Why predict that the growth rate is going to slow now? Simple. Because current models have already been trained on pretty much the entire meaningful part of the Internet. Where are they going to get more data?
The exponential growth part of the curve was largely based on being able to fit more and more training data into the models. Now that all the meaningful training data has been fed in, further growth will come from one of two things: generating training data from one LLM to feed into another one (dangerous, highly likely to lead to "down the rabbit hole forever" hallucinations, and weeding those out is a LOT of work and will therefore contribute to slower growth), or else finding better ways to tweak the models to make better use of the available training data (which will produce growth, but much slower than what "Hey, we can slurp up the entire Internet now!" was producing in terms of rate of growth).
And yes, there is more training data available because the Internet is not static: the Internet of 2025 has more meaningful, human-generated content than the Internet of 2024. But it also has a lot more AI-generated content, which will lead into the rabbit-hole problem where one AI's hallucinations get baked into the next one's training, so the extra data that can be harvested from the 2025 Internet is almost certainly going to produce slower growth in meaningful results (as opposed to hallucinated results).
> The point is, why predict that the growth rate is going to slow exactly now? What evidence are you going to look at?
Why predict that the (absolute) growth rate is going to keep accelerating past exactly now?
Exponential growth always assumes a constant relative growth rate, which works in the fiction of economics, but is otherwise far from an inevitability. People like to point to Moore's law ad nauseam, but other things like "the human population" or "single-core performance" keep accelerating until they start cooling off.
> And note, there are good reasons to predict a speedup, too; as models get more intelligent, they will be able to accelerate the R&D process.
And if heaven forbid, R&D ever turns out to start taking more work for the same marginal returns on "ability to accelerate the process", then you no longer have an exponential curve. Or for that matter, even if some parts can be accelerated to an amazing extent, other parts may get strung up on Amdahl's law.
It's fine to predict continued growth, and it's even fine to predict that a true inflection point won't come any time soon, but exponential growth is something else entirely.
I think progress per dollar spent has actually slowed dramatically over the last three years. The models are better, but AI spending has increased by several orders of magnitude during the same time, from hundreds of millions to hundreds of billions. You can only paper over the lack of fundamental progress by spending on more compute for so long. And even if you manage to keep up the current capex, there certainly isn't enough capital in the world to accelerate spending for very long.
> why predict that the growth rate is going to slow exactly now?
why predict that it will continue? Nobody ever actually makes an argument that growth is likely to continue, they just extrapolate from existing trends and make a guess, with no consideration of the underlying mechanics.
Oh, go on then, I'll give a reason: this bubble is inflated primarily by venture capital, and is not profitable. The venture capital is starting to run out, and there is no convincing evidence that the businesses will become profitable.
Indeed you can't be sure. But on the other hand a bunch of the commentariat has been claiming (with no evidence) that we're at the midpoint of the sigmoid for the last three years. They were wrong. And then you had the AI frontier lab insiders who predicted an accelerating pace of progress for the last three years. They were right. Now, the frontier labs rarely (never?) provide evidence either, but they do have about a year of visibility into the pipeline, unlike anyone outside.
So at least my heuristic is to wait until a frontier lab starts warning about diminishing returns and slowdowns before calling the midpoint or multiple labs start winding down capex. The first component might have misaligned incentives, but if we're in a realistic danger of hitting a wall in the next year, the capex spending would not be accelerating the way it is.
Capex requirements might be on a different curve than model improvements.
E.g. you might need to accelerate spending to get sub-linear growth in model output.
If valuations depend on hitting the curves described in the article, you might see accelerating capex at precisely the time improvements are dropping off.
I don’t think frontier labs are going to be a trustworthy canary. If Anthropic says they’re reaching the limit and OpenAI holds the line that AGI is imminent, talent and funding will flee Anthropic for OpenAI. There’s a strong incentive to keep your mouth shut if things aren’t going well.
> Indeed you can't be sure. But on the other hand a bunch of the commentariat has been claiming (with no evidence) that we're at the midpoint of the sigmoid for the last three years.
I haven’t followed things closely, but I’ve seen more statements that we may be near the midpoint of a sigmoid than that we are at it.
> Thy were wrong. And then you had the AI frontier lab insiders who predicted an accelerating pace of progress for the last three years. They were right.
I know it’s an unfair question because we don’t have an objective way to measure speed of progress in this regard, but do you have evidence for models not only getting better, but getting better faster? (Remember: even at the midpoint of a sigmoid, there still is significant growth)
There are a few other limitations, in particular how much energy, hardware and funding we (as a society) can afford to throw at the problem, as well as the societal impact.
AI development is currently given a free pass on these points, but it's very unclear how long that will last. Regardless of scientific and technological potential, I believe that we'll hit some form of limit soon.
> I'm sure people were saying that about commercial airline speeds in the 1970's too.
Or CPU frequencies in the 1990's. Also we spent quite a few decades at the end of the 19th century thinking that physics was finished.
I'm not sure that explaining it as an "S curve" is really the right metaphor either, though.
You get the "exponential" growth effect when there's a specific technology invented that "just needs to be applied", and the application tricks tend to fall out quickly. For sure generative AI is on that curve right now, with everyone big enough to afford a datacenter training models like there's no tomorrow and feeding a community of a million startups trying to deploy those models.
But nothing about this is modeled correctly as an "exponential", except in the somewhat trivial sense of "the community of innovators grows like a disease as everyone hops on board". Sure, the petri dish ends up saturated pretty quickly and growth levels off, but that's not really saying much about the problem.
There's a Mulla Nasrudin joke that's sort of relevant here:
Nasrudin is on a flight, when suddenly the pilot comes on the intercom, saying, "Passengers, we apologize, but we have experienced an engine burn-out. The plane can still fly on the remaining three engines, but we'll be delayed in our arrival by two hours."
Nasrudin speaks up "let's not worry, what's 2 hours really"
A few minutes later, the airplane shakes, and passengers see smoke coming out of another engine. Again, the intercom crackles to life.
"This is your captain speaking. Apologies, but due to a second engine burn-out, we'll be delayed by another two hours."
The passengers are agitated, but the Mulla once again tries to remains calm.
Suddenly, the third engine catches fire. Again, the pilot comes on the intercom and says, "I know you're all scared, but this is a very advanced aircraft, and it can safely fly on only a single engine. But we will be delayed by yet another two hours."
At this, Nasrudin shouts, "This is ridiculous! If one more engine goes, we'll be stuck up here all day"
Progress in information systems cannot be compared to progress in physical systems.
For starters, physical systems compete for limited resources and labor.
For another, progress in software vastly reduces the cost of improved designs. Whereas progress in physical systems can enable but still increase the cost of improved designs.
Finally, the underlying substrate of software is digital hardware, which has been improving in both capabilities and economics exponentially for almost 100 years.
Looking at information systems as far back as the first coordination of differentiating cells to human civilization is one of exponential improvement. Very slow, slow, fast, very fast. (Can even take this further, to first metabolic cycles, cells, multi-purpose genes, modular development genes, etc. Life is the reproduction of physical systems via information systems.)
Same with human technological information systems, from cave painting, writing, printing, telegraph, phone, internet, etc.
It would be VERY surprising if AI somehow managed to fall off the exponential information system growth path. Not industry level surprising, but "everything we know about how useful information compounds" level surprising.
> Looking at information systems as far back as the first coordination of differentiating cells to human civilization is one of exponential improvement.
Under what metric? Most of the things you mention don't have numerical values to plot on a curve. It's a vibe exponential, at best.
Life and humans have become better and better at extracting available resources and energy, but there's a clear limit to that (100%) and the distribution of these things in the universe is a given, not something we control. You don't run information systems off empty space.
>[..] to first metabolic cycles, cells, multi-purpose genes, modular development genes, etc.
One example is when cells discovered energy production using mitochondria. Mitochondria add new capabilities to the cell, with (almost) no downside like: weight, temperature-sensitivity, pressure-sensitivity. It's almost 100% upside.
If someone tried to predict the future number of mitochondria-enabled cells from the first one, he could be off by 10^20 less cells.
I am writing a story the last 20 days, with that exact story plot, have to get my stuff together and finish it.
That's fallacious reasoning, you are extrapolating from survivorship bias. A lot of technologies, genes, or species have failed along the way. You are also subjectively attributing progression as improvements, which is problematic as well, if you speak about general trends. Evolution selects for adaptation not innovation. We use the theory of evolution to explain the emergence of complexity, but that's not the sole direction and there are many examples where species evolved towards simplicity (again).
Resource expense alone could be the end of AI. You may look up historic island populations, where technological demands (e.g. timber) usually led to extinction by resource exhaustion and consequent ecosystem collapse (e.g. deforestation leading to soil erosion).
> But a lot of technologies turn out to be S-shaped, not purely exponential, because there are limiting factors.
I'd argue all of them. Any true exponential eventually gets to a point where no computer can even store its numerical value. It's a physically absurd curve.
The narrative quietly assumes that this exponential curve can in fact continue since it will be the harbinger of the technological singularity. Seems more than a bit eschatological, but who knows.
If we suppose this tech rapture does happen, all bets are off; in that sense it's probably better to assume the curve is sigmoidal, since the alternative is literally beyond human comprehension.
I am getting the sense that the 2nd deriative of the curve is already hitting negative teritory. models get updated, and I don't feel I'm getting better answers from the LLMs.
On the application front though, it feels that the advancements from a couple of years ago are just beginning to trickle down to product space. I used to do some video editing as a hobby. Recently I picked it up again, and was blown away by how much AI has chipped away the repetitive stuff, and even made attempts at the more creative aspects of production, with mixed but promising results.
And, maybe I'm missing something, but to me it seems obvious that flat top part of the S curve is going to be somewhere below human ability... because, as you say, of the training data. How on earth could we train an LLM to be smarter than us, when 100% of the material we use to teach it how to think, is human-style thinking?
Maybe if we do a good job, only a little bit below human ability -- and what an accomplishment that would still be!
But still -- that's a far cry from the ideas espoused in articles like this, where AI is just one or two years away from overtaking us.
The standard way to do this is Reinforcement Learning: we do not teach the model how to do the task, we let it discover the _how_ for itself and only grade it based on how well it did, then reinforce the attempts where it did well. This way the model can learn wildly superhuman performance, e.g. it's what we used to train AlphaGo and AlphaZero.
Yes. It's true that we don't know, with any certainty, (1) whether we are hitting limits to growth intrinsic to current hardware and software, (2) whether we will need new hardware or software breakthroughs to continue improving models, and (3) what the timing of any necessary breakthroughs, because innovation doesn't happen on a predictable schedule. There are unknown unknowns.[a]
However, there's no doubt that at a global scale, we're sure trying to maintain current rates of improvement in AI. I mean, the scale and breadth of global investment dedicated to improving AI, presently, is truly unprecedented. Whether all this investment is driven by FOMO or by foresight, is irrelevant. The underlying assumption in all cases is the same: We will figure out, somehow, how to overcome all known and unknown challenges along the way. I have no idea what the odds of success may be, but they're not zero. We sure live in interesting times!
>> it would be extremely surprising if these improvements suddenly stopped.
> But a lot of technologies turn out to be S-shaped, not purely exponential, because there are limiting factors.
An S-curve is exactly the opposite of "suddenly" stopping.
It is possible for us to get a sudden stop, due to limiting factors.
For a hypothetical: if Moore's Law had continued until we hit atomic resolution instead of the slowdown as we got close to it, that would have been an example of a sudden stop: can't get transistors smaller than atoms, but yet it would have been possible (with arbitrarily large investments that we didn't have) to halve transistor sizes every 18 months until suddenly we can't.
Now I think about it, the speed of commercial airlines is also an example of a sudden stop: we had to solve sonic booms first before even considering a Concorde replacement.
The cost of the next number in a GPT (3>4>5) seems to be in 2 ways:
1) $$$
2) data
The second (data) also isn't cheap. As it seems we've already gotten through all the 'cheap' data out there. So much so that synthetic data (fart huffing) is a big thing now. People tell it's real and useful and passes the glenn-horf theore... blah blah blah.
So it really more so comes down to just:
1) $$$^2 (but really pick any exponent)
In that, I'm not sure this thing is a true sigmoid curve (see: biology all the time). I think it's more a logarithmic cost here. In that, it never really goes away, but it gets really expensive to carry out for large N.
[To be clear, lots of great shit happens out there in large N. An AI god still may lurk in the long slow slope of $N, the cure for boredom too, or knowing why we yawn, etc.]
"I'm sure people were saying that about commercial airline speeds in the 1970's too."
But there are others that keep going also. Moore's law is still going (mostly, slowing), and made it past a few pinch points where people thought it was the end.
The point is, that over 30 decades, many people said Moore's law was at an end, and then it wasn't, there was some breakthrough that kept it going. Maybe a new one will happen.
The thing with AI is, maybe the S curve flattens out , after all the jobs are gone.
Everyone is hoping the S curve flattens out somewhere just below human level, but what if it flattens out just beyond human level? We're still screwed.
Each specific technology can be S-shaped, but advancements in achieving goals can still maintain an exponential curve. e.g. Moore's law is dead with the end of Dennard scaling, but computation improvements still happen with parallelism.
Meta's Behemoth shows that scaling number of parameters has diminished returns, but we still have many different ways to continue advancements. Those who point at one thing and say "see", isn't really seeing. Of course there are limits, like energy but with nuclear energy or photon-based computing were nowhere near the limits.
> But a lot of technologies turn out to be S-shaped, not purely exponential, because there are limiting factors.
S curves are exponential before they start tapering off though. It's hard to predict how long that could continue, so there's an argument to be made that we should remain optimistic and milk that while we can lest pessimism cut off investment top early.
> But a lot of technologies turn out to be S-shaped, not purely exponential, because there are limiting factors.
S curves are exponential before they start tapering off though. It's hard to predict how long that could continue, so there's an argument to be made that we should milk that while we can.
Ironically, given that it probably mistakes a sigmoid curve for an exponential curve, "Failing to understand the exponential, again" is an extremely apt name for this blog post.
> I'm sure people were saying that about commercial airline speeds in the 1970's too.
They'd be wrong, of course - for not realizing demand is a limiting factor here. Airline speeds plateaued not because we couldn't make planes go faster anymore, but because no one wanted them to go faster.
This is partially economical and partially social factor - transit times are bucketed by what they enable people to do. It makes little difference if going from London to New York takes 8 hours instead of 12 - it's still in the "multi-day business trip" bucket (even 6 hours goes into that bucket, once you add airport overhead). Now, if you could drop that to 3 hours, like Concorde did[0], that finally moves it into "hop over for a meet, fly back the same day" bucket, and then business customers start paying attention[1].
For various technical, legal and social reasons, we didn't manage to cross that chasm before money for R&D dried out. Still, the trend continued anyway - in military aviation and, later, in supersonic missiles.
With AI, the demand is extreme and only growing, and it shows no sign of being structured into classes with large thresholds between them - in fact, models are improving faster than we're able to put them to any use; even if we suddenly hit a limit now and couldn't train even better models anymore, we have decades of improvements to extract just from learning how to properly apply the models we have. But there's no sign we're about to hit a wall with training any time soon.
Airline speeds are inherently a bad example for the argument you're making, but in general, I don't think pointing out S-curves is all that useful. As you correctly observe:
> But a lot of technologies turn out to be S-shaped, not purely exponential, because there are limiting factors.
But, what happens when one technology - or rather, one metric of that technology - stops improving? Something else starts - another metric of that technology, or something built on top of it, or something that was enabled by it. The exponent is S-curves on top of S-curves, all the way down, but how long that exponent is depends on what you consider in scope. So, a matter of accounting. So yeah, AI progress can flatten tomorrow or continue exponentially for the next couple years - depending on how narrowly you define "AI progress".
[1] - This is why Elon Musk wasn't immediately laughed out of the room after proposing using Starship for moving people and cargo across the Earth, back in 2017. Hopping between cities on an ICBM sounds borderline absurd for many reasons, but it also promised cutting flight time to less than one hour between any two points on Earth, which put it a completely new bucket, even more interesting for businesses.
There’s a key way to think about a process that looks exponential and might or might not flatten out into an S curve: reasoning about fundamental limits. For COVID it would obviously flatten out because there are finite humans, and it did when the disease had in fact infected most humans on the planet. For commercial airlines you could reason about the speed of sound or escape velocity and see there is again a natural upper limit- although which of those two would dominate would have very different real world implications.
For computational intelligence, we have one clear example of an upper limit in a biological human brain. It only consumes about 25W and has much more intelligence than today’s LLMs in important ways. Maybe that’s the wrong limit? But Moore’s law has been holding for a very long time. And smart physicists like Feynman in his seminal lecture predicting nanotechnology in 1959 called “there’s plenty of room at the bottom” have been arguing that we are extremely far from running into any fundamental physical limits on the complexity of manufactured objects. The ability to manufacture them we presume is limited by ingenuity, which jokes aside shows no signs of running out.
Training data is a fine argument to consider. Especially since there are training on “the whole internet” sorta. The key breakthrough of transformers wasn’t in fact autoregressive token processing or attention or anything like that. It was that they can learn from (memorize / interpolate between / generalize) arbitrary quantities of training data. Before that every kind of ML model hit scaling limits pretty fast. Resnets got CNNs to millions of parameters but they still became quite difficult to train. Transformers train reliably on every size data set we have ever tried with no end in sight. The attention mechanism shortens the gradient path for extremely large numbers of parameters, completely changing the rules of what’s possible with large networks. But what about the data to feed them?
There are two possible counter arguments there. One is that humans don’t need exabytes of examples to learn the world. You might reasonably conclude from this that NNs have some fundamental difference vs people and that some hard barrier of ML science innovation lies in the way. Smart scientists like Yann LeCun would agree with you there. I can see the other side of that argument too - that once a system is capable of reasoning and learning it doesn’t need exhaustive examples to learn to generalize. I would argue that RL reasoning systems like GRPO or GSPO do exactly this - they let the system try lots of ways to approach a difficult problem until they figure out something that works. And then they cleverly find a gradient towards whatever technique had relative advantage. They don’t need infinite examples of the right answer. They just need a well chosen curriculum of difficult problems to think about for a long time. (Sounds a lot like school.) Sometimes it takes a very long time. But if you can set it up correctly it’s fairly automatic and isn’t limited by training data.
The other argument is what the Silicon Valley types call “self play” - the goal of having an LLM learn from itself or its peers through repeated games or thought experiments. This is how Alpha Go was trained, and big tech has been aggressively pursuing analogs for LLMs. This has not been a runaway success yet. But in the area of coding agents, arguably where AI is having the biggest economic impact right now, self play techniques are an important part of building both the training and evaluation sets. Important public benchmarks here start from human curated examples and algorithmically enhance them to much larger sizes and levels of complexity. I think I might have read about similar tricks in math problems but I’m not sure. Regardless it seems very likely that this has a way to overcome any fundamental limit on availability of training data as well, based on human ingenuity instead.
Also, if the top of the S curve is high enough, it doesn’t matter that it’s not truly exponential. The interesting stuff will happen before it flattens out. E.g. COVID. Consider the y axis “human jobs replaced by AI” instead of “smartness” and yes it’s obviously an S curve.
> For computational intelligence, we have one clear example of an upper limit in a biological human brain. It only consumes about 25W and has much more intelligence than today’s LLMs in important ways. Maybe that’s the wrong limit?
It's a good reference point, but I see no reason for it to be an upper limit - by the very nature of how biological evolution works, human brains are close to the worst possible brains advanced enough to start a technological revolution. We're the first brain on Earth that crossed that threshold, and in evolutionary timescales, all that followed - all human history - happened in an instant. Evolution didn't have time yet to iterate on our brain design.
> But a lot of technologies turn out to be S-shaped, not purely exponential, because there are limiting factors.
S curves are exponential before they start tapering off though. It's hard to predict how long that could continue, so there's an argument to be made that we should remain optimistic and milk that while we can lest pessimism cut off investment too early.
As they say, every exponential is a sigmoid in disguise. I think the exponential phase of growth for LLM architectures is drawing to a close, and fundamentally new architectures will be necessary for meaningful advances.
I'm also not convinced by the graphs in this article. OpenAI is notoriously deceptive with their graphs, and as Gary Marcus has already noted, that METR study comes with a lot of caveats: [https://garymarcus.substack.com/p/the-latest-ai-scaling-grap...]
Exponential curves don't last for long fortunately, or the universe would have turned into a quark soup. The example of COVID is especially ironic, considering it stopped being a real concern within 3 years of its advent despite the exponential growth in the early years.
Those who understand exponentials should also try to understand stock and flow.
Reminds me a bit of the "ultraviolet catastrophe".
> The ultraviolet catastrophe, also called the Rayleigh–Jeans catastrophe, was the prediction of late 19th century and early 20th century classical physics that an ideal black body at thermal equilibrium would emit an unbounded quantity of energy as wavelength decreased into the ultraviolet range.
[...]
> The phrase refers to the fact that the empirically derived Rayleigh–Jeans law, which accurately predicted experimental results at large wavelengths, failed to do so for short wavelengths.
Right. Nobody believed that the intensity would go to infinity. What they believed was that the theory was incomplete, but they didn't know how or why. And the solution required inventing a completely new theory.
Exponentials exist in their environment. Didn't Covid stop because we ran out of people to infect. Of course it can't keep going exponential, because there aren't exponential people to infect.
What is this limit on AI? It is technology, energy, something. All these things can be over-come, to keep the exponential going.
And of course, systems also break at the exponential. Maybe AI is stopped by the world economy collapsing. AI advancement would be stopped, but that is cold comfort to the humans.
Data. Think of our LLMs like bacteria in a Petri dish. When first introduced, they achieve exponential growth by rapidly consuming the dish's growth medium. Once the medium is consumed, growth slows and then stops.
The corpus of information on the Internet, produced over several decades, is the LLM's growth medium. And we're not producing new growth medium at an exponential rate.
> What is this limit on AI? It is technology, energy, something. All these things can be over-come, to keep the exponential going.
That's kind of begging the question. Obviously if all the limitations on AI can be overcome growth would be exponential. Even the biggest ai skeptic would agree. The question is, will it?
Long COVID is still a thing, the nAbs immunity is pretty paltry because the virus keeps changing its immunity profile so much. T-cells help but also damage the host because of how COVID overstimulates them. A big reason people aren't dying like they used to is because of the government's strategy of constant infection which boosts immunity regularly* while damaging people each time, that plus how Omicron changed SARS-CoV-2's cell entry mechanism to avoid cell-cell fusion (syncytia) that caused huge over-reaction in lung tissue.
It's possible to understand both exponential and limiting behavior at the same time. I work in an office full of scientists. Our team scrammed the workplace on March 10, 2020.
To the scientists, it was intuitively obvious that the curve could not surpass 100% of the population. An exponential curve with no turning point is almost always seen as a sure sign that something is wrong with your model. But we didn't have a clue as to the actual limit, and any putative limit below 100% would need a justification, which we didn't have, or some dramatic change to the fundamental conditions, which we couldn't guess.
The typical practice is to watch the curve for any sign of a departure from exponential behavior, and then say: "I told you so." ;-)
The first change may have been social isolation. In fact that was pretty much the only arrow in our quivers. The second change was the vaccine, which changed both the infection rate and the mortality rate, dramatically.
I'm curious as to whether the consensus is that the observed behaviour of COVID waves was ever fully and satisfactorily explained - the tend to grow exponentially but then seemingly saturate at a much lower point than a naïve look at the curve might suggest?
Stopped being a concern primarily due to heavy vaccination campaigns though. It is still raging, just not nearly as many people are dying. The immunity from infection these days is pretty paltry.
> By the end of 2027, models will frequently outperform experts on many tasks.
In passing the quiz-es
> Models will be able to autonomously work for full days (8 working hours) by mid-2026.
Who will carry responsibility for the consequences of these model's errors? What tools will be avaiable to that resposible _person_?
--
Tehchno optimists will be optimistic. Techno pessimists will be pessimistic.
Processes we're discussing have their own limiting factors which no one mentiones. Why to mention what exactly makes graph go up and holds it from going exponential? Why to mention or discuss inherit limitations of the LLMs architecture? Or what is legal perspective on AI agency?
Thus we're discussing results of AI models passing tests and people's perception of other people opinions.
You don't actually need to have a "responsible person"; you can just have an AI do stuff. It might make a mistake; the only difference between that and an employee is that you can't punish an AI. If you're any good at management and not a psychopath, the ability to have someone to punish for mistakes isn't actually important
The importance of having a human be responsible is about alignment. We have a fundamental belief that human beings are comprehensible and have goals that are not completely opaque. That is not true of any piece of software. In the case of deterministic software, you can’t argue with a bug. It doesn’t matter how many times you tell it that no, that’s not what either the company or the user intended, the result will be the same.
With an AI, the problem is more subtle. The AI may absolutely be able to understand what you’re saying, and may not care at all, because its goals are not your goals, and you can’t tell what its goals are. Having a human be responsible bypasses that. The point is not to punish the AI, the point is to have a hope to stop it from doing things that are harmful.
I will worry when I see Startups competing on products with companies 10x, 100x, or 1000x times their size. Like a small team producing a Photoshop replacement. So far I haven't seen anything like that. Big companies don't seem to be launching new products faster either, or fixing some of their products that have been broken for a long time (MS teams...)
AI obviously makes some easy things much faster, maybe helps with boilerplate, we still have to see this translate into real productivity.
I think the real turning point is when there isn’t the need for something like photoshop. Creatives that I speak to yearn for the day when they can stop paying the adobe tax.
There will always be an adobe tax so to speak. Creatives want high quality and reliable tools to be able to produce high quality things.
I could imagine a world where a small team + AI creates an open source tool that is better than current day Photoshop. However if that small team has that power, so does adobe, and what we perceive as "good" or "high quality" will shift.
It's interesting that he brings up the example of "exponential" growth in the case of COVID infections even though it was actually logistic growth[1] that saturates once resources get exhausted. What makes AI different?
Readit News
Their Covid example is a great counter argument to their point in that covid isn’t still growing exponentially.
Where the AI skeptics (or even just pragmatists, like myself) chime in is saying “yeah AI will improve. But LLMs are a limited technology that cannot fully bridge the gap between what they’re producing now, and what the “hypists” claim they’ll be able to do in the future.”
People like Sam Altman know ChatGPT is a million miles away from AGI. But their primary goal is to make money. So they have to convince VCs that their technology has a longer period of exponential growth than what it actually will have.
The argument is not that it will keep growing exponentially forever (obviously that is physically impossible), rather that:
- given a sustained history of growth along a very predictable trajectory, the highest likelihood short term scenario is continued growth along the same trajectory. Sample a random point on an s-curve and look slightly to the right, what’s the most common direction the curve continues?
- exponential progress is very hard to visualize and see, it may appear to hardly make any progress while far away from human capabilities, then move from just below to far above human very quickly
The problem isn’t that exponential growth is hard to visualise. The problem is that LLMs, as advanced and useful a technique as it is, isn’t suited for AGI and thus will never get us even remotely to the stage of AGI.
The human like capabilities are really just smoke and mirrors.
It’s like when people anthropomorphisise their car; “she’s being temperamental today”. Except we know the car is not intelligence and it’s just a mechanical problem. Whereas it’s in the AI tech firms best interest to upsell the human-like characteristics of LLMs because that’s how they get VC money. And as we know, building and running models isn’t cheap.
Exponential growth over the short term is very uninteresting. Exponential growth is exciting when it can compound.
E.g. if i offered you an investing opportunity 500% / per year compounded daily - that's amazing. If the fine print is that that rate will only last for the very near term (say a week), then it would be worse than a savings account.
Which is pretty ironic given the title of the post
Both things can be true, since they're orthogonal.
Having AI do these things was complete fiction 10 years ago. And after 5 years of LLM AI, people do start to see serious limits and stunted growth with the current LLM approaches, while also seeing that nobody has proposed another serious contended to that approach.
Similarly, going to the moon was science finction 100 years ago. And yet, we're now not only not in Mars, but 50+ years without a new moon manned landing. Same for airplanes. Science fiction in 1900. Mostly stale innovation wise for the last 30 years.
A lot of curves can fit an exponential line plot, without the progress going forward being exponential.
We would have 1 trillion transistor cpus following Moore's "exponential curve"
[1] https://en.m.wikipedia.org/wiki/Transistor_count
Cerebras wafer scale chip has 4 trillion transistors.
https://www.cerebras.ai/chip
It is also, notably, _wafer-scale_. The metric is not just "number of transistors", but in fact "number of transistors per cm2"
And really, we’ve had even less than that. The first large scale reasoning model was o1, which was released 12 months ago. More useful coding agents are even newer than that. This narrative that we’ve been using these tools for many years and are now hitting a wall doesn’t match my experience at all. AI-assisted coding is way better than it was a year ago, let alone five.
Why would it be "at most" 3? We had Chat GPT commercially available as private beta API on 2020. It's only the mass public that got 3.5 3 years ago.
But those who'd do the noticing as per my argument is not just Joe Public (which could be oblivious), but people already starting in 2020, and includes people working in the space, who worked with LLM and LLM-like architectures 2-3 years before 2020.
I'm sure people were saying that about commercial airline speeds in the 1970's too.
But a lot of technologies turn out to be S-shaped, not purely exponential, because there are limiting factors.
With LLM's at the moment, the limiting factors might turn out to be training data, cost, or inherent limits of the transformer approach and the fact that LLM's fundamentally cannot learn outside of their context window. Or a combination of all of these.
The tricky thing about S curves is, you never know where you are on them until the slowdown actually happens. Are we still only in the beginning of the growth part? Or the middle where improvement is linear rather than exponential? And then the growth starts slowing...
Yes of course it’s not going to increase exponentially forever.
The point is, why predict that the growth rate is going to slow exactly now? What evidence are you going to look at?
It’s possible to make informed predictions (eg “Moore’s law can’t get you further than 1nm with silicon due to fundamental physical limits”). But most commenters aren’t basing their predictions in anything as rigorous as that.
And note, there are good reasons to predict a speedup, too; as models get more intelligent, they will be able to accelerate the R&D process. So quality per-researcher is now proportional to the exponential intelligence curve, AND quantity of researchers scales with number of GPUs (rather than population growth which is much slower).
It’s likely that it will slow down at some point, but the highest likelihood scenario for the near future is that scaling will continue.
Why predict that the growth rate is going to slow now? Simple. Because current models have already been trained on pretty much the entire meaningful part of the Internet. Where are they going to get more data?
The exponential growth part of the curve was largely based on being able to fit more and more training data into the models. Now that all the meaningful training data has been fed in, further growth will come from one of two things: generating training data from one LLM to feed into another one (dangerous, highly likely to lead to "down the rabbit hole forever" hallucinations, and weeding those out is a LOT of work and will therefore contribute to slower growth), or else finding better ways to tweak the models to make better use of the available training data (which will produce growth, but much slower than what "Hey, we can slurp up the entire Internet now!" was producing in terms of rate of growth).
And yes, there is more training data available because the Internet is not static: the Internet of 2025 has more meaningful, human-generated content than the Internet of 2024. But it also has a lot more AI-generated content, which will lead into the rabbit-hole problem where one AI's hallucinations get baked into the next one's training, so the extra data that can be harvested from the 2025 Internet is almost certainly going to produce slower growth in meaningful results (as opposed to hallucinated results).
Why predict that the (absolute) growth rate is going to keep accelerating past exactly now?
Exponential growth always assumes a constant relative growth rate, which works in the fiction of economics, but is otherwise far from an inevitability. People like to point to Moore's law ad nauseam, but other things like "the human population" or "single-core performance" keep accelerating until they start cooling off.
> And note, there are good reasons to predict a speedup, too; as models get more intelligent, they will be able to accelerate the R&D process.
And if heaven forbid, R&D ever turns out to start taking more work for the same marginal returns on "ability to accelerate the process", then you no longer have an exponential curve. Or for that matter, even if some parts can be accelerated to an amazing extent, other parts may get strung up on Amdahl's law.
It's fine to predict continued growth, and it's even fine to predict that a true inflection point won't come any time soon, but exponential growth is something else entirely.
why predict that it will continue? Nobody ever actually makes an argument that growth is likely to continue, they just extrapolate from existing trends and make a guess, with no consideration of the underlying mechanics.
Oh, go on then, I'll give a reason: this bubble is inflated primarily by venture capital, and is not profitable. The venture capital is starting to run out, and there is no convincing evidence that the businesses will become profitable.
So at least my heuristic is to wait until a frontier lab starts warning about diminishing returns and slowdowns before calling the midpoint or multiple labs start winding down capex. The first component might have misaligned incentives, but if we're in a realistic danger of hitting a wall in the next year, the capex spending would not be accelerating the way it is.
E.g. you might need to accelerate spending to get sub-linear growth in model output.
If valuations depend on hitting the curves described in the article, you might see accelerating capex at precisely the time improvements are dropping off.
I don’t think frontier labs are going to be a trustworthy canary. If Anthropic says they’re reaching the limit and OpenAI holds the line that AGI is imminent, talent and funding will flee Anthropic for OpenAI. There’s a strong incentive to keep your mouth shut if things aren’t going well.
I haven’t followed things closely, but I’ve seen more statements that we may be near the midpoint of a sigmoid than that we are at it.
> Thy were wrong. And then you had the AI frontier lab insiders who predicted an accelerating pace of progress for the last three years. They were right.
I know it’s an unfair question because we don’t have an objective way to measure speed of progress in this regard, but do you have evidence for models not only getting better, but getting better faster? (Remember: even at the midpoint of a sigmoid, there still is significant growth)
Progress has most definitely not been happening at an _accelerating_ pace.
Deleted Comment
AI development is currently given a free pass on these points, but it's very unclear how long that will last. Regardless of scientific and technological potential, I believe that we'll hit some form of limit soon.
Or CPU frequencies in the 1990's. Also we spent quite a few decades at the end of the 19th century thinking that physics was finished.
I'm not sure that explaining it as an "S curve" is really the right metaphor either, though.
You get the "exponential" growth effect when there's a specific technology invented that "just needs to be applied", and the application tricks tend to fall out quickly. For sure generative AI is on that curve right now, with everyone big enough to afford a datacenter training models like there's no tomorrow and feeding a community of a million startups trying to deploy those models.
But nothing about this is modeled correctly as an "exponential", except in the somewhat trivial sense of "the community of innovators grows like a disease as everyone hops on board". Sure, the petri dish ends up saturated pretty quickly and growth levels off, but that's not really saying much about the problem.
Nasrudin is on a flight, when suddenly the pilot comes on the intercom, saying, "Passengers, we apologize, but we have experienced an engine burn-out. The plane can still fly on the remaining three engines, but we'll be delayed in our arrival by two hours."
Nasrudin speaks up "let's not worry, what's 2 hours really"
A few minutes later, the airplane shakes, and passengers see smoke coming out of another engine. Again, the intercom crackles to life.
"This is your captain speaking. Apologies, but due to a second engine burn-out, we'll be delayed by another two hours."
The passengers are agitated, but the Mulla once again tries to remains calm.
Suddenly, the third engine catches fire. Again, the pilot comes on the intercom and says, "I know you're all scared, but this is a very advanced aircraft, and it can safely fly on only a single engine. But we will be delayed by yet another two hours."
At this, Nasrudin shouts, "This is ridiculous! If one more engine goes, we'll be stuck up here all day"
For starters, physical systems compete for limited resources and labor.
For another, progress in software vastly reduces the cost of improved designs. Whereas progress in physical systems can enable but still increase the cost of improved designs.
Finally, the underlying substrate of software is digital hardware, which has been improving in both capabilities and economics exponentially for almost 100 years.
Looking at information systems as far back as the first coordination of differentiating cells to human civilization is one of exponential improvement. Very slow, slow, fast, very fast. (Can even take this further, to first metabolic cycles, cells, multi-purpose genes, modular development genes, etc. Life is the reproduction of physical systems via information systems.)
Same with human technological information systems, from cave painting, writing, printing, telegraph, phone, internet, etc.
It would be VERY surprising if AI somehow managed to fall off the exponential information system growth path. Not industry level surprising, but "everything we know about how useful information compounds" level surprising.
Under what metric? Most of the things you mention don't have numerical values to plot on a curve. It's a vibe exponential, at best.
Life and humans have become better and better at extracting available resources and energy, but there's a clear limit to that (100%) and the distribution of these things in the universe is a given, not something we control. You don't run information systems off empty space.
> For starters, physical systems compete for limited resources and labor.
> Finally, the underlying substrate of software is digital hardware…
See how these are related?
One example is when cells discovered energy production using mitochondria. Mitochondria add new capabilities to the cell, with (almost) no downside like: weight, temperature-sensitivity, pressure-sensitivity. It's almost 100% upside.
If someone tried to predict the future number of mitochondria-enabled cells from the first one, he could be off by 10^20 less cells.
I am writing a story the last 20 days, with that exact story plot, have to get my stuff together and finish it.
Resource expense alone could be the end of AI. You may look up historic island populations, where technological demands (e.g. timber) usually led to extinction by resource exhaustion and consequent ecosystem collapse (e.g. deforestation leading to soil erosion).
I'd argue all of them. Any true exponential eventually gets to a point where no computer can even store its numerical value. It's a physically absurd curve.
If we suppose this tech rapture does happen, all bets are off; in that sense it's probably better to assume the curve is sigmoidal, since the alternative is literally beyond human comprehension.
On the application front though, it feels that the advancements from a couple of years ago are just beginning to trickle down to product space. I used to do some video editing as a hobby. Recently I picked it up again, and was blown away by how much AI has chipped away the repetitive stuff, and even made attempts at the more creative aspects of production, with mixed but promising results.
Also elegantly formulated by: https://idlewords.com/talks/web_design_first_100_years.htm
And, maybe I'm missing something, but to me it seems obvious that flat top part of the S curve is going to be somewhere below human ability... because, as you say, of the training data. How on earth could we train an LLM to be smarter than us, when 100% of the material we use to teach it how to think, is human-style thinking?
Maybe if we do a good job, only a little bit below human ability -- and what an accomplishment that would still be!
But still -- that's a far cry from the ideas espoused in articles like this, where AI is just one or two years away from overtaking us.
The standard way to do this is Reinforcement Learning: we do not teach the model how to do the task, we let it discover the _how_ for itself and only grade it based on how well it did, then reinforce the attempts where it did well. This way the model can learn wildly superhuman performance, e.g. it's what we used to train AlphaGo and AlphaZero.
However, there's no doubt that at a global scale, we're sure trying to maintain current rates of improvement in AI. I mean, the scale and breadth of global investment dedicated to improving AI, presently, is truly unprecedented. Whether all this investment is driven by FOMO or by foresight, is irrelevant. The underlying assumption in all cases is the same: We will figure out, somehow, how to overcome all known and unknown challenges along the way. I have no idea what the odds of success may be, but they're not zero. We sure live in interesting times!
---
[a] https://en.wikipedia.org/wiki/There_are_unknown_unknowns
> But a lot of technologies turn out to be S-shaped, not purely exponential, because there are limiting factors.
An S-curve is exactly the opposite of "suddenly" stopping.
It is possible for us to get a sudden stop, due to limiting factors.
For a hypothetical: if Moore's Law had continued until we hit atomic resolution instead of the slowdown as we got close to it, that would have been an example of a sudden stop: can't get transistors smaller than atoms, but yet it would have been possible (with arbitrarily large investments that we didn't have) to halve transistor sizes every 18 months until suddenly we can't.
Now I think about it, the speed of commercial airlines is also an example of a sudden stop: we had to solve sonic booms first before even considering a Concorde replacement.
1) $$$
2) data
The second (data) also isn't cheap. As it seems we've already gotten through all the 'cheap' data out there. So much so that synthetic data (fart huffing) is a big thing now. People tell it's real and useful and passes the glenn-horf theore... blah blah blah.
So it really more so comes down to just:
1) $$$^2 (but really pick any exponent)
In that, I'm not sure this thing is a true sigmoid curve (see: biology all the time). I think it's more a logarithmic cost here. In that, it never really goes away, but it gets really expensive to carry out for large N.
[To be clear, lots of great shit happens out there in large N. An AI god still may lurk in the long slow slope of $N, the cure for boredom too, or knowing why we yawn, etc.]
But there are others that keep going also. Moore's law is still going (mostly, slowing), and made it past a few pinch points where people thought it was the end.
The point is, that over 30 decades, many people said Moore's law was at an end, and then it wasn't, there was some breakthrough that kept it going. Maybe a new one will happen.
The thing with AI is, maybe the S curve flattens out , after all the jobs are gone.
Everyone is hoping the S curve flattens out somewhere just below human level, but what if it flattens out just beyond human level? We're still screwed.
Meta's Behemoth shows that scaling number of parameters has diminished returns, but we still have many different ways to continue advancements. Those who point at one thing and say "see", isn't really seeing. Of course there are limits, like energy but with nuclear energy or photon-based computing were nowhere near the limits.
if the money dries up because the investors lose faith on the exponential continuing, then his future looks much dimmer
S curves are exponential before they start tapering off though. It's hard to predict how long that could continue, so there's an argument to be made that we should remain optimistic and milk that while we can lest pessimism cut off investment top early.
S curves are exponential before they start tapering off though. It's hard to predict how long that could continue, so there's an argument to be made that we should milk that while we can.
They were also saying that about CPU clock speeds.
They'd be wrong, of course - for not realizing demand is a limiting factor here. Airline speeds plateaued not because we couldn't make planes go faster anymore, but because no one wanted them to go faster.
This is partially economical and partially social factor - transit times are bucketed by what they enable people to do. It makes little difference if going from London to New York takes 8 hours instead of 12 - it's still in the "multi-day business trip" bucket (even 6 hours goes into that bucket, once you add airport overhead). Now, if you could drop that to 3 hours, like Concorde did[0], that finally moves it into "hop over for a meet, fly back the same day" bucket, and then business customers start paying attention[1].
For various technical, legal and social reasons, we didn't manage to cross that chasm before money for R&D dried out. Still, the trend continued anyway - in military aviation and, later, in supersonic missiles.
With AI, the demand is extreme and only growing, and it shows no sign of being structured into classes with large thresholds between them - in fact, models are improving faster than we're able to put them to any use; even if we suddenly hit a limit now and couldn't train even better models anymore, we have decades of improvements to extract just from learning how to properly apply the models we have. But there's no sign we're about to hit a wall with training any time soon.
Airline speeds are inherently a bad example for the argument you're making, but in general, I don't think pointing out S-curves is all that useful. As you correctly observe:
> But a lot of technologies turn out to be S-shaped, not purely exponential, because there are limiting factors.
But, what happens when one technology - or rather, one metric of that technology - stops improving? Something else starts - another metric of that technology, or something built on top of it, or something that was enabled by it. The exponent is S-curves on top of S-curves, all the way down, but how long that exponent is depends on what you consider in scope. So, a matter of accounting. So yeah, AI progress can flatten tomorrow or continue exponentially for the next couple years - depending on how narrowly you define "AI progress".
Ergo, not all that useful.
--
[0] - https://simpleflying.com/concorde-fastest-transatlantic-cros...
[1] - This is why Elon Musk wasn't immediately laughed out of the room after proposing using Starship for moving people and cargo across the Earth, back in 2017. Hopping between cities on an ICBM sounds borderline absurd for many reasons, but it also promised cutting flight time to less than one hour between any two points on Earth, which put it a completely new bucket, even more interesting for businesses.
Deleted Comment
For computational intelligence, we have one clear example of an upper limit in a biological human brain. It only consumes about 25W and has much more intelligence than today’s LLMs in important ways. Maybe that’s the wrong limit? But Moore’s law has been holding for a very long time. And smart physicists like Feynman in his seminal lecture predicting nanotechnology in 1959 called “there’s plenty of room at the bottom” have been arguing that we are extremely far from running into any fundamental physical limits on the complexity of manufactured objects. The ability to manufacture them we presume is limited by ingenuity, which jokes aside shows no signs of running out.
Training data is a fine argument to consider. Especially since there are training on “the whole internet” sorta. The key breakthrough of transformers wasn’t in fact autoregressive token processing or attention or anything like that. It was that they can learn from (memorize / interpolate between / generalize) arbitrary quantities of training data. Before that every kind of ML model hit scaling limits pretty fast. Resnets got CNNs to millions of parameters but they still became quite difficult to train. Transformers train reliably on every size data set we have ever tried with no end in sight. The attention mechanism shortens the gradient path for extremely large numbers of parameters, completely changing the rules of what’s possible with large networks. But what about the data to feed them?
There are two possible counter arguments there. One is that humans don’t need exabytes of examples to learn the world. You might reasonably conclude from this that NNs have some fundamental difference vs people and that some hard barrier of ML science innovation lies in the way. Smart scientists like Yann LeCun would agree with you there. I can see the other side of that argument too - that once a system is capable of reasoning and learning it doesn’t need exhaustive examples to learn to generalize. I would argue that RL reasoning systems like GRPO or GSPO do exactly this - they let the system try lots of ways to approach a difficult problem until they figure out something that works. And then they cleverly find a gradient towards whatever technique had relative advantage. They don’t need infinite examples of the right answer. They just need a well chosen curriculum of difficult problems to think about for a long time. (Sounds a lot like school.) Sometimes it takes a very long time. But if you can set it up correctly it’s fairly automatic and isn’t limited by training data.
The other argument is what the Silicon Valley types call “self play” - the goal of having an LLM learn from itself or its peers through repeated games or thought experiments. This is how Alpha Go was trained, and big tech has been aggressively pursuing analogs for LLMs. This has not been a runaway success yet. But in the area of coding agents, arguably where AI is having the biggest economic impact right now, self play techniques are an important part of building both the training and evaluation sets. Important public benchmarks here start from human curated examples and algorithmically enhance them to much larger sizes and levels of complexity. I think I might have read about similar tricks in math problems but I’m not sure. Regardless it seems very likely that this has a way to overcome any fundamental limit on availability of training data as well, based on human ingenuity instead.
Also, if the top of the S curve is high enough, it doesn’t matter that it’s not truly exponential. The interesting stuff will happen before it flattens out. E.g. COVID. Consider the y axis “human jobs replaced by AI” instead of “smartness” and yes it’s obviously an S curve.
It's a good reference point, but I see no reason for it to be an upper limit - by the very nature of how biological evolution works, human brains are close to the worst possible brains advanced enough to start a technological revolution. We're the first brain on Earth that crossed that threshold, and in evolutionary timescales, all that followed - all human history - happened in an instant. Evolution didn't have time yet to iterate on our brain design.
S curves are exponential before they start tapering off though. It's hard to predict how long that could continue, so there's an argument to be made that we should remain optimistic and milk that while we can lest pessimism cut off investment too early.
I'm also not convinced by the graphs in this article. OpenAI is notoriously deceptive with their graphs, and as Gary Marcus has already noted, that METR study comes with a lot of caveats: [https://garymarcus.substack.com/p/the-latest-ai-scaling-grap...]
Those who understand exponentials should also try to understand stock and flow.
> The ultraviolet catastrophe, also called the Rayleigh–Jeans catastrophe, was the prediction of late 19th century and early 20th century classical physics that an ideal black body at thermal equilibrium would emit an unbounded quantity of energy as wavelength decreased into the ultraviolet range.
[...]
> The phrase refers to the fact that the empirically derived Rayleigh–Jeans law, which accurately predicted experimental results at large wavelengths, failed to do so for short wavelengths.
https://en.wikipedia.org/wiki/Ultraviolet_catastrophe
What is this limit on AI? It is technology, energy, something. All these things can be over-come, to keep the exponential going.
And of course, systems also break at the exponential. Maybe AI is stopped by the world economy collapsing. AI advancement would be stopped, but that is cold comfort to the humans.
Data. Think of our LLMs like bacteria in a Petri dish. When first introduced, they achieve exponential growth by rapidly consuming the dish's growth medium. Once the medium is consumed, growth slows and then stops.
The corpus of information on the Internet, produced over several decades, is the LLM's growth medium. And we're not producing new growth medium at an exponential rate.
Gulf money, for one. DoD budget would be another.
Booms are economic phenomena, not technological phenomena. When looking for a limiting factor of a boom, think about the money taps.
That's kind of begging the question. Obviously if all the limitations on AI can be overcome growth would be exponential. Even the biggest ai skeptic would agree. The question is, will it?
If you think COVID isn't still around: https://www.cdc.gov/nwss/rv/COVID19-national-data.html
* one might call this strategy forced vaccination with a known dangerous live vaccine strain lol
To the scientists, it was intuitively obvious that the curve could not surpass 100% of the population. An exponential curve with no turning point is almost always seen as a sure sign that something is wrong with your model. But we didn't have a clue as to the actual limit, and any putative limit below 100% would need a justification, which we didn't have, or some dramatic change to the fundamental conditions, which we couldn't guess.
The typical practice is to watch the curve for any sign of a departure from exponential behavior, and then say: "I told you so." ;-)
The first change may have been social isolation. In fact that was pretty much the only arrow in our quivers. The second change was the vaccine, which changed both the infection rate and the mortality rate, dramatically.
In passing the quiz-es
> Models will be able to autonomously work for full days (8 working hours) by mid-2026.
Who will carry responsibility for the consequences of these model's errors? What tools will be avaiable to that resposible _person_?
--
Tehchno optimists will be optimistic. Techno pessimists will be pessimistic.
Processes we're discussing have their own limiting factors which no one mentiones. Why to mention what exactly makes graph go up and holds it from going exponential? Why to mention or discuss inherit limitations of the LLMs architecture? Or what is legal perspective on AI agency?
Thus we're discussing results of AI models passing tests and people's perception of other people opinions.
With an AI, the problem is more subtle. The AI may absolutely be able to understand what you’re saying, and may not care at all, because its goals are not your goals, and you can’t tell what its goals are. Having a human be responsible bypasses that. The point is not to punish the AI, the point is to have a hope to stop it from doing things that are harmful.
AI obviously makes some easy things much faster, maybe helps with boilerplate, we still have to see this translate into real productivity.
I could imagine a world where a small team + AI creates an open source tool that is better than current day Photoshop. However if that small team has that power, so does adobe, and what we perceive as "good" or "high quality" will shift.
[1] https://en.wikipedia.org/wiki/Logistic_function#Modeling_ear...