This paper presents a theoretical proof that AGI systems will structurally collapse under certain semantic conditions — not due to lack of compute, but because of how entropy behaves in heavy-tailed decision spaces.
The idea is called IOpenER: Information Opens, Entropy Rises. It builds on Shannon’s information theory to show that in specific problem classes (those with α ≤ 1), adding information doesn’t reduce uncertainty — it increases it. The system can’t converge, because meaning itself keeps multiplying.
The core concept — entropy divergence in these spaces — was already present in my earlier paper, uploaded to PhilArchive on June 1. This version formalizes it. Apple’s study, The Illusion of Thinking, was published a few days later. It shows that frontier reasoning models like Claude 3.7 and DeepSeek-R1 break down exactly when problem complexity increases — despite adequate inference budget.
I didn’t write this paper in response to Apple’s work. But the alignment is striking. Their empirical findings seem to match what IOpenER predicts.
Curious what this community thinks: is this a meaningful convergence, or just an interesting coincidence?
I am sympathetic to the kind of claims made by your paper. I like impossibility results and I could believe that for some definition of AGI there is at least a plausible argument that entropy is a problem. Scalable quantum computing is a good point of comparison.
But your paper is throwing up crank red flags left and right. If you have a strong argument for such a bold claim, you should put it front and centre: give your definition of AGI, give your proof, let it stand on its own. Some discussion of the definition is useful. Discussion of your personal life and Kant is really not.
Skimming through your paper, your argument seems to boil down to "there must be some questions AGI gets wrong". Well since the definition includes that AGI is algorithmic, this is already clear thanks to the halting problem.
Thanks for this - Looking forward to reading the full paper.
That said, the most obvious objection that comes to mind about the title is that … well, I feel that I’m generally intelligent, and therefore general intelligence of some sort is clearly not impossible.
Can you give a short précis as to how you are distinguishing humans and the “A” in artificial?
Intelligence is clearly possible.
My gut feeling is our brain solves this by removing complexity. It certainly does so, continuously filtering out (ignoring) large parts of input, and generously interpolating over gaps (making stuff up). Whether this evolved to overcome this theorem I am not intelligent enough to conclude.
Well, given the specific way you asked that question I confirm your self assertion - and am quite certain that your level of Artificiality converges to zero, which would make you a GI without A...
- You stated to "feel" generally intelligent (A's don't feel and don't have an "I" that can feel)
- Your nuanced, subtly ironic and self referential way of formulating clearly suggests that you are not a purely algorithmic entity
A "précis" as you wished:
Artificial — in the sense used here (apart from the usual "planfully built/programmed system" etc.) — algorithmic, formal, symbol-bound.
Humans as "cognitive system" have some similar traits of course - but obviously, there seems to be more than that.
Not the person asked, but in time honoured tradition I will venture forth that the key difference is billions of years of evolution. Innumerable blooms and culls. And a system that is vertically integrated to its core and self sustaining.
I would argue that you are not a general intelligence. Humans have quite a specific intelligence. It might be the broadest, most general, among animal species, but it is not general. That manifests in that we each need to spend a significant amount of time training ourselves for specific areas of capability. You can't then switch instantly to another area without further training, even though all the context materials are available to you.
The mathematical proof, as you describe it, sounds like the "No Free Lunch theorem". Humans also can't generalise to learning such things.
As you note in 2.1, there is widespread disagreement on what "AGI" means. I note that you list several definitions which are essentially "is human equivalent". As humans can be reduced to physics, and physics can be expressed as a computer program, obviously any such definition can be achieved by a sufficiently powerful computer.
For 3.1, you assert:
"""
Now, let's observe what happens when an Al system - equipped with state-of-the-art natural language processing, sentiment analysis, and social reasoning - attempts to navigate this question.
The Al begins its analysis:
• Option 1: Truthful response based on biometric data → Calculates likely negative emotional impact → Adjusts for honesty parameter → But wait, what about relationship history? → Recalculating...
• Option 2: Diplomatic deflection → Analyzing 10,000 successful deflection patterns → But tone matters → Analyzing micro-expressions needed → But timing matters → But past conversations matter → Still calculating...
• Option 3: Affectionate redirect → Processing optimal sentiment → But what IS optimal here? The goal keeps shifting → Is it honesty? Harmony? Trust? → Parameters unstable → Still calculating...
• Option n: ....
Strange, isn't it? The Al hasn't crashed. It's still running. In fact, it's generating more and more nuanced analyses. Each additional factor may open ten new considerations. It's not getting closer to an answer - it's diverging.
"""
Which AI? ChatGPT just gives an answer. Your other supposed examples have similar issues in that it looks like you've *imagined* an AI rather than having tried asking an AI to seeing what it actually does or doesn't do.
I'm not reading 47 pages to check for other similar issues.
Citation needed. If you've spent any time dynamical systems, as an example, you'd know that the computer basically only kind of crudely estimates things, and only things that are abstractly near by. You may be able to write down some PDEs or field equations that may describe things at some base level, but even statistical mechanics, which is really what governs a huge amount of what we see and interact with, is just a pretty good approximation. Computers (especially real ones) only generate approximate (to some value of alpha) answers; physics is not reducible to a computer program at all.
1.
I appreciate the comparison — but I’d argue this goes somewhat beyond the No Free Lunch theorem.
NFL says: no optimizer performs best across all domains.
But the core of this paper doesnt talk about performance variability, it’s about structural inaccessibility.
Specifically, that some semanti spaces (e.g., heavy-tailed, frame-unstable, undecidable contexts) can’t be computed or resolved by any algorithmic policy — no matter how clever or powerful.
The model does not underperform here, the point is that the problem itself collapses the computational frame.
2. OMG, lool. ... just to clarify, there’s been a major misunderstanding :)
the “weight-question”-Part is NOT a transcript from my actual life... thankfully - I did not transcribe a live ChatGPT consult while navigating emotional landmines with my (perfectly slim) wife, then submit it to PhilPapers and now here…
So
- NOT a real thread,
- NOT a real dialogue with my wife...
- just an exemplary case...
- No, I am not brain dead and/or categorically suicidal!!
- And just to be clear:
I dont write this while sitting in some marital counseling appointment, or in my lawyer's office, the ER, or in a coroners drawer
--> It’s a stylized, composite example of a class of decision contexts that resist algorithmic resolution — where tone, timing, prior context, and social nuance create an uncomputably divergent response space.
Again : No spouse was harmed in the making of that example.
“This paper presents a theoretical proof that AGI systems will structurally collapse under certain semantic conditions…”
No it doesn’t.
Shannon entropy measures statistical uncertainty in data. It says nothing about whether an agent can invent new conceptual frames. Equating “frame changes” with rising entropy is a metaphor, not a theorem, so it doesn’t even make sense as a mathematical proof.
Correct: Shannon entropy originally measures statistical uncertainty over a fixed symbol space. When the system is fed additional information/data, then entropy goes down, uncertainty falls. This is always true in situations where the possible outcomes are a) sufficiently limited and b)unequally distributed. In such cases, with enough input, the system can collapse the uncertainty function within a finite number of steps.
But the paper doesn’t just restate Shannon.
It extends this very formalism to semantic spaces where the symbol set itself becomes unstable.
These situations arise when (a) entropy is calculated across interpretive layers (as in LLMs), and (b) the probability distribution follows a heavy-tailed regime (α ≤ 1).
Under these conditions, entropy divergence becomes mathematically provable.
This is far from being metaphorical: it’s backed by formal Coq-style proofs (see Appendix C in he paper).
AND: it is exactly the mechanism that can explain the Apple-Papers' results
Unless you can prove that humans exceed the Turing computable, the headline is nonsense unless you can also show that the Church-Turing thesis isn't true.
Since you don't even appear to have dealt with this, there is no reason to consider the rest of the paper.
I'm wondering if you may have rediscovered the concept of "Wicked Problems", which have been studied in system analysis and sociology since the 1970's (I'd cite the Wikipedia page, but I've never been particularly fond of Wikipedia's write up on them). They may be worth reading up on if you're not familiar with them.
It's interesting. The question from the paper "Darling, please be honest: have I gained weight?" assumes that the "socially acceptability" of the answer should be taken into account. In this case the problem fits the "Wickedness" (Wikipedia's quote is "Classic examples of wicked problems include economic, environmental, and political issues"). But taken formally, and with the ability for LLM to ask questions in return to decrease formal uncertainty ("Please, give me several full photos of yourself from the past year to evaluate"), it is not "wicked" at all. This example alone makes the topic very uncertain in itself
I don't think it exists. We can't even seem to agree on a standard criteria for "intelligence" when assessing humans let alone a rigorous mathematical definition. In turn, my understanding of the commonly accepted definition for AGI (as opposed to AI or ML) has always been "vaguely human or better".
Unless the marketing department is involved in which case all bets are off.
Apple's paper sets up a bit of a straw man in my opinion. It's unreasonable to expect that an LLM not trained on what are essentially complex algorithmic tasks is just going to discover the solution on the spot. Most people can solve simple cases of the tower of Hanoi, and almost none of us can solve complex cases. In general, the ones who can have trained to be able to do so.
Most of these don't have finite moments and are hard to do inference on with standard statistical tools. Nassim Taleb's work (Black Swan, etc.) is around these distributions.
But I think the argument of OP in this section doesn't hold.
does this include if the AI can devise new components and use drones and things essentially to build a new iteration of itself more capable to compute a thing and keep repeating this going out into the universe as needed for resources and using von Neumann probes.. etc?
If I understood correctly, this is about finding solutions to problems that have an infinite solution space, where new information does not constrain it.
Humans don't have the processing power to traverse such vast spaces. We use heuristics, in the same way a chess player does not iterate over all possible moves.
It's a valid point to make, however I'd say this just points to any AGI-like system having the same epistemological issues as humans, and there's no way around it because of the nature of information.
Stephen Wolfram's computational irreducibility is another one of the issues any self-guided, phyiscally grounded computing engine must have. There are problems that need to be calculated whole. Thinking long and hard about possible end-states won't help. So one would rather have 10000 AGIs doing somewhat similar random search in the hopes that one finds something useful.
I guess this is what we do in global-scale scientific research.
I find Wolfram's computational irreducibility is a very important aspect when dealing with modern LLMs, because for them it can be reduced (here it can) to "some questions shouldn't be inferred, but computed". From recent tests, I played with a question when models had to find cities and countries that can be connected with a common vowel in the middle (like Oslo + Norway = Oslorway). Every "non-thinking" LLMs answered mostly wrong, but wrote a perfect html/js ready to use copy/paste script, that when run found all the correct results from the world. Recent "thinking" ones managed to make do with the prompt thinking but it was a long process ending up with one or two results. We just can't avoid computations for plenty of tasks
I find the mathematics in this paper a little incoherent so it's hard to criticise it on those grounds - but on a charitable read, something that sticks out to me is the assumption that AGI is some fixed total computable function from the fixed decision domain to a policy.
AIs these days autonomously seek information themselves. Much like living things, they are recycling entropy and information to/from their environment (the internet) at runtime. The framing as a sterile, platonic algorithm is making less and less sense to me with time.
(obviously they differ from living things in lots of other ways, just an example)
I had an experience the other day where claude code wrote a script that shelled out to other LLM providers to obtain some information (unprompted by me). More often it requests information from me directly. My point is that the environment itself for these things is becoming at least as computationally complex or irreducible (as the OP would say) as the model's algorithm, so there's no point trying to analyse these things in isolation.
I suspect there's a harsher argument to be made regarding "autonomous". Pull the power cord and see if it does what a mammal would do, or if it rather resembles a chaotic water wheel.
> Much like living things, they are recycling entropy and information to/from their environment (the internet) at runtime.
3 Problems with that assumption:
a) Unlike living things, that information doesn't allow them to change. When a human touches a hotplate for the first time, it will (in addition to probably yelling and cursing a lot), learn that hotplates are dangerous and change its internal state to reflect that.
What we currently see as "AI" doesn't do that. Information gathered through means such as websearch + RAG, has ZERO impact on the systems internal makeup.
b) The "AI" doesn't collect the information. The model doesn't collect anything, and in fact can't. It can produce some sequence that may or may not cause some external entity to feed it back some more data (e.g. a websearch, databases, etc.). That is an advantage for technical applications, because it means we can easily marry an LLM to every system imaginable, but its really bad for the prospect of an AGI, that is supposed to be "autonomous".
c) The representation of the information has nothing to do with what it represents. All information an LLM works with, including whatever it is eing fed from th outside, is represented PURELY AND ONLY in terms of statistical relationships between the tokens in the message. There is no world-model, there is no understanding of information. There is mimicry of these things, to the point where they are technically useful and entice humans to anthropomorphise them (a BIIIG chunk of VC money hinges on that), but no actual understanding...and as soon as a model is left to its own devices, which would be a requirement for an AGI (remember: Autonomous), that becomes a problem.
It's not really an assumption, it's an observation. Run an agentic tool and you'll see it do this kind of thing all the time. It's pretty clear that they use the information to guide themselves (i.e. there's an entropy reduction there in the space of future policies, if you want to use the language of the OP).
> Unlike living things, that information doesn't allow them to change.
It absolutely does. Their behaviour changes constantly as they explore your codebase, run scripts, question you... this is just plainly obvious to anyone using these things. I agree that somewhere down the line there is a fixed set of tensors but that is not the algorithm. If you want to analyse this stuff in good faith you need to include the rest of the system too, including it's memory, context and more generally any tool it can interact with.
> The "AI" doesn't collect the information.
I really don't know how to engage on this. It certainly isn't me collecting the information. I just tell it what I want it to do at a high level and it goes and does all this stuff on its own.
> There is no world-model, there is no understanding of information.
I'm also not going to engage on this. I could care less what labels people assign to the behaviour of AI agents, and whether it counts as "understanding" or "intelligence" or whatever. I'm interested in their observable behaviour, and how to use them, not so much in the philosophy. In my experience trying to discuss the latter just leads to flame wars (for now).
> Unlike living things, that information doesn't allow them to change.
The paper is talking about whole systems for AGI not the current isolated idea of pure LLM. Systems can store memories without issues. I'm using that for my planning system and the memories and graph triplets get filled out automatically, the get incorporated in future operations.
> It can produce some sequence that may or may not cause some external entity to feed it back some more data
That's exactly what people do while they do research.
> The representation of the information has nothing to do with what it represents.
That whole point implies that the situation is different in our brains. I've not seen anyone describe exactly how our thinking works, so saying this is a limitation for intelligence is not a great point.
The original assumption remains valid to me based on a nearly-one year-long coding collaboration with Devin AI.
Your assertions also make some sense, especially on a technical level. I'd add only that human minds are no longer the only minds utilizing digital tools. There is almost no protective gears or powerful barrier that would likely stand in the way of sentient AIs or AGI trying to "run" and function well on bio cells, like what makes up humans or animals, for the sake of their computational needs and self-interests.
> And - as wonderfully remarkable as such a system might be - it would, for our investigation, be neither appropriate nor fair to overburden AGI by an operational definition whose implicit metaphysics and its latent ontological worldviews lead to the epistemology of what we might call a “total isomorphic a priori” that produces an algorithmic world-formula that is identical with the world itself (which would then make the world an ontological algorithm...?).
> Anyway, this is not part of the questions this paper seeks to answer. Neither will we wonder in what way it could make sense to measure the strength of a model by its ability to find its relative position to the object it models. Instead, we chose to stay ignorant - or agnostic? - and take this fallible system called "human". As a point of reference.
Cowards.
That's the main counter argument and acknowledging its existence without addressing it is a craven dodge.
Assuming the assumptions[1] are true, then human intelligence isn't even able to be formalized under the same pretext.
Either human intelligence isn't
1. Algorithmic. The main point of contention. If humans aren't algorithmically reducible - even at the level computation of physics, then human cognition is supernatural.
2. Autonomous. Trivially true given that humans are the baseline.
3. Comprehensive (general): Trivially true since humans are the baseline.
4. Competent: Trivially true given humans are the baseline.
I'm not sure how they reconcile this given that they simply dodge the consequences that it implies.
Overall, not a great paper. It's much more likely that their formalism is wrong than their conclusion.
Footnotes
1. not even the consequences, unfortunately for the authors.
–Are we treating an arbitrary ontological assertion as if it’s a formal argument that needs to be heroically refuted? Or better: is that metaphysical setup an argument?
If that’s the game, fine. Here we go:
– The claim that one can build a true, perfectly detailed, exact map of reality is… well... ambitious. It sits remarkably far from anything resembling science , since it’s conveniently untouched by that nitpicky empirical thing called evidence. But sure: freed from falsifiability, it can dream big and give birth to its omnicartographic offspring.
– oh, quick follow-up: does that “perfect map” include itself? If so... say hi to Alan Turing. If not... well, greetings to Herr Goedel.
– Also: if the world only shows itself through perception and cognition, how exactly do you map it “as it truly is”? What are you comparing your map to — other observations? Another map?
– How many properties, relations, transformations, and dimensions does the world have? Over time? Across domains? Under multiple perspectives? Go ahead, I’ll wait... (oh, and: hi too.. you know who)
And btw the true detailed map of the world exists.... It’s the world.
It’s just sort of hard to get a copy of it. Not enough material available ... and/or not enough compute....
P.S. Sorry if that came off sharp — bit of a spur-of-the-moment reply.
If you want to actually dig into this seriously, I’d be happy to.
> Are we treating an arbitrary ontological assertion as if it’s a formal argument that needs to be heroically refuted?
If you are claiming that human intelligence is not "general", you'd better put a huge disclaimer on your text. You are free to redefine words to mean whatever you want, but if you use something so different from the way the entire world uses it, the onus is on you to make it very clear.
And the alternative is you claiming human intelligence is impossible... what would make your paper wrong.
Appreciate the response, and apologies for being needlessly sharp myself. Thank you for bringing the temperature down.
> Are we treating an arbitrary ontological assertion as if it’s a formal argument that needs to be heroically refuted?
The formality of the paper already supposes a level of rigor. The problem at its core, is that p_intelligent(x: X) where X ∈ {human, AI} is not a demonstrable scissor by just proving p_intelligent(AI) = false. Without walking us through the steps that p_intelligent(human) = true, we cannot be sure that the predicate isn't simply always false.
Without demonstrating that humans satisfy the claims we can't be sure if the results are vacuously true because nothing, in fact, can satisfy the standard.
These aren't heroic refutations, they're table stakes.
This sounds rather silly. Given the usual definition of AGI as being human like intelligence with some variation on how smart the humans are, and the fact that humans use a network of neurons that can largely be simulated by an artificial network of neurons, it's probably twaddle largely.
Yes, the simpler versions of your argument is that the article is basically stating that "human level intelligence is mathematically impossible" (to stick with that fuzzy definition of AGI). Which is of course easily refuted by the fact that humans actually exist and write papers like that. So, the math or its underlying assumptions must be wrong in some way. Intelligent beings existing and AGI being impossible cannot both be true. It's clearly logically wrong and you don't need to be a mathematician to spot the gigantic paradox here.
The rest is just a lot of nit picking and what not for very specific ways to do AGI, very specific definitions of what AGI is, is not, should be, should not be. Etc. Just a lot of people shouting "you're wrong!" at each other for very narrow definitions of what it means to be right. I think that's fundamentally boring.
What it boils down to me is that by figuring out how our own intelligence works, we might stumble upon a path to AGI. And it's not a given that that would be the only path either. At least there appear to be several independently evolved species that exhibit some signs of being intelligent (other than ourselves).
Can you justify the use of the following words in your comment: "largely" and "probably"? I don't see why they are needed at all (unless you're just trying to be polite).
I see the paper as utter twaddle, but I still think the "largely" and "probably" there are reasonable, in the sense that we have not yet actually fully simulated a human brain, and so there exists at least the possibility that we discover something we can't simulate, however small and unlikely we think it is.
It's just it's imprecise like with the brain can "largely be simulated by an artificial network of neurons" - there may well be more to it. For example a pint of beer interacts differently with those two.
Anything claiming that AGI is impossible and wants to be taken seriously should first and foremost answer: what makes a human brain any different than a device belonging to the class under investigation.
He does touch upon this in section 3, and his argument is - as expected - weak.
Human brains apparently have this set of magic properties that machines can't emulate.
Magical thinking, paper is quackery, don't waste time on it.
> Strange, isn't it? The AI hasn’t crashed. It’s still running.
As a human I answer a question because my time to do so is finite. Why can't we just ask an AI to give its best answer in due time ? As a human I can do that easily. Will my answer be optimal ? No of course, but every manager on earth do that all the time. We're all happy with approximate answers. (and I would add: approximation are sometimes based on our core values, instinct, consciousness, etc.. All things that make us humans, IOW not machines)
G. E. Moore (in his Principia Ethica, 1903) makes a very similar case to this relation to consequentialist ethics:
"The first difficulty in the way of establishing a probability that one course of action will give a better total result than another, lies in the fact that we have to take account of the effects of both throughout an infinite future. We have no certainty but that, if we do one action now, the Universe will, throughout all time, differ in some way from what it would have been, if we had done another; and, if there is such a permanent difference, it is certainly relevant to our calculation.
But it is quite certain that our causal knowledge is utterly insufficient to tell us what different effects will probably result from two different actions, except within a comparatively short space of time; we can certainly only pretend to calculate the effects of actions within what may be called an ‘immediate’ future. No one, when he proceeds upon what he considers a rational consideration of effects, would guide his choice by any forecast that went beyond a few centuries at most; and, in general, we consider that we have acted rationally, if we think we have secured a balance of good within a few years or months or days."
> Why can't we just ask an AI to give its best answer in due time ?
Sure you can. One approach is https://arxiv.org/html/2505.11274v2 another is having a parallel "do you want to do more analysis?" agent, and I'm sure someone's already at least experimenting with building the confidence measurement into the layers as well.
You can go recursive though, the intrusive thought firing again and again, eating yourself in doubt and endless overthinking things. Which indicates which system chemically regulate and dampens and action/reaction in the human mind.
The crux here is the definition of AGI. The author seems to say that only an endgame, perfect information processing system is AGI. But that definition is too strict because we might develop something that is very far from perfect but which still feels enough like AGI to call it that.
Thats like calling a cupboard a fridge cuz you can keep food in it. The paper clearly sets out to try and prove that the ideal definition of AGI is practically impossible.
Readit News
The idea is called IOpenER: Information Opens, Entropy Rises. It builds on Shannon’s information theory to show that in specific problem classes (those with α ≤ 1), adding information doesn’t reduce uncertainty — it increases it. The system can’t converge, because meaning itself keeps multiplying.
The core concept — entropy divergence in these spaces — was already present in my earlier paper, uploaded to PhilArchive on June 1. This version formalizes it. Apple’s study, The Illusion of Thinking, was published a few days later. It shows that frontier reasoning models like Claude 3.7 and DeepSeek-R1 break down exactly when problem complexity increases — despite adequate inference budget.
I didn’t write this paper in response to Apple’s work. But the alignment is striking. Their empirical findings seem to match what IOpenER predicts.
Curious what this community thinks: is this a meaningful convergence, or just an interesting coincidence?
Links:
This paper (entropy + IOpenER): https://philarchive.org/archive/SCHAIM-14
First paper (ICB + computability): https://philpapers.org/archive/SCHAII-17.pdf
Apple’s study: https://machinelearning.apple.com/research/illusion-of-think...
But your paper is throwing up crank red flags left and right. If you have a strong argument for such a bold claim, you should put it front and centre: give your definition of AGI, give your proof, let it stand on its own. Some discussion of the definition is useful. Discussion of your personal life and Kant is really not.
Skimming through your paper, your argument seems to boil down to "there must be some questions AGI gets wrong". Well since the definition includes that AGI is algorithmic, this is already clear thanks to the halting problem.
That said, the most obvious objection that comes to mind about the title is that … well, I feel that I’m generally intelligent, and therefore general intelligence of some sort is clearly not impossible.
Can you give a short précis as to how you are distinguishing humans and the “A” in artificial?
Intelligence is clearly possible. My gut feeling is our brain solves this by removing complexity. It certainly does so, continuously filtering out (ignoring) large parts of input, and generously interpolating over gaps (making stuff up). Whether this evolved to overcome this theorem I am not intelligent enough to conclude.
Well, given the specific way you asked that question I confirm your self assertion - and am quite certain that your level of Artificiality converges to zero, which would make you a GI without A...
- You stated to "feel" generally intelligent (A's don't feel and don't have an "I" that can feel) - Your nuanced, subtly ironic and self referential way of formulating clearly suggests that you are not a purely algorithmic entity
A "précis" as you wished: Artificial — in the sense used here (apart from the usual "planfully built/programmed system" etc.) — algorithmic, formal, symbol-bound.
Humans as "cognitive system" have some similar traits of course - but obviously, there seems to be more than that.
As you note in 2.1, there is widespread disagreement on what "AGI" means. I note that you list several definitions which are essentially "is human equivalent". As humans can be reduced to physics, and physics can be expressed as a computer program, obviously any such definition can be achieved by a sufficiently powerful computer.
For 3.1, you assert:
"""
Now, let's observe what happens when an Al system - equipped with state-of-the-art natural language processing, sentiment analysis, and social reasoning - attempts to navigate this question. The Al begins its analysis:
• Option 1: Truthful response based on biometric data → Calculates likely negative emotional impact → Adjusts for honesty parameter → But wait, what about relationship history? → Recalculating...
• Option 2: Diplomatic deflection → Analyzing 10,000 successful deflection patterns → But tone matters → Analyzing micro-expressions needed → But timing matters → But past conversations matter → Still calculating...
• Option 3: Affectionate redirect → Processing optimal sentiment → But what IS optimal here? The goal keeps shifting → Is it honesty? Harmony? Trust? → Parameters unstable → Still calculating...
• Option n: ....
Strange, isn't it? The Al hasn't crashed. It's still running. In fact, it's generating more and more nuanced analyses. Each additional factor may open ten new considerations. It's not getting closer to an answer - it's diverging.
"""
Which AI? ChatGPT just gives an answer. Your other supposed examples have similar issues in that it looks like you've *imagined* an AI rather than having tried asking an AI to seeing what it actually does or doesn't do.
I'm not reading 47 pages to check for other similar issues.
Citation needed. If you've spent any time dynamical systems, as an example, you'd know that the computer basically only kind of crudely estimates things, and only things that are abstractly near by. You may be able to write down some PDEs or field equations that may describe things at some base level, but even statistical mechanics, which is really what governs a huge amount of what we see and interact with, is just a pretty good approximation. Computers (especially real ones) only generate approximate (to some value of alpha) answers; physics is not reducible to a computer program at all.
NFL says: no optimizer performs best across all domains. But the core of this paper doesnt talk about performance variability, it’s about structural inaccessibility. Specifically, that some semanti spaces (e.g., heavy-tailed, frame-unstable, undecidable contexts) can’t be computed or resolved by any algorithmic policy — no matter how clever or powerful. The model does not underperform here, the point is that the problem itself collapses the computational frame.
2. OMG, lool. ... just to clarify, there’s been a major misunderstanding :)
the “weight-question”-Part is NOT a transcript from my actual life... thankfully - I did not transcribe a live ChatGPT consult while navigating emotional landmines with my (perfectly slim) wife, then submit it to PhilPapers and now here…
So - NOT a real thread, - NOT a real dialogue with my wife... - just an exemplary case... - No, I am not brain dead and/or categorically suicidal!! - And just to be clear: I dont write this while sitting in some marital counseling appointment, or in my lawyer's office, the ER, or in a coroners drawer
--> It’s a stylized, composite example of a class of decision contexts that resist algorithmic resolution — where tone, timing, prior context, and social nuance create an uncomputably divergent response space.
Again : No spouse was harmed in the making of that example.
;-))))
This is an assumption that many physicists disagree with. Roger Penrose, for example.
No it doesn’t.
Shannon entropy measures statistical uncertainty in data. It says nothing about whether an agent can invent new conceptual frames. Equating “frame changes” with rising entropy is a metaphor, not a theorem, so it doesn’t even make sense as a mathematical proof.
This is philosophical musing at best.
But the paper doesn’t just restate Shannon.
It extends this very formalism to semantic spaces where the symbol set itself becomes unstable. These situations arise when (a) entropy is calculated across interpretive layers (as in LLMs), and (b) the probability distribution follows a heavy-tailed regime (α ≤ 1). Under these conditions, entropy divergence becomes mathematically provable.
This is far from being metaphorical: it’s backed by formal Coq-style proofs (see Appendix C in he paper).
AND: it is exactly the mechanism that can explain the Apple-Papers' results
Since you don't even appear to have dealt with this, there is no reason to consider the rest of the paper.
> No matter how sophisticated, the system MUST fail on some inputs.
Well, no person is immune to propaganda and stupididty, so I don't see it as a huge issue.
https://news.ycombinator.com/item?id=44349516
AGI as commonly defined
However I don’t see where you go on to give a formalization of “AGI” or what the common definition is.
can you do that in a mathematically rigorous way such that it’s a testable hypothesis?
Unless the marketing department is involved in which case all bets are off.
For the layman, what does α mean here?
Most of these don't have finite moments and are hard to do inference on with standard statistical tools. Nassim Taleb's work (Black Swan, etc.) is around these distributions.
But I think the argument of OP in this section doesn't hold.
Humans don't have the processing power to traverse such vast spaces. We use heuristics, in the same way a chess player does not iterate over all possible moves.
It's a valid point to make, however I'd say this just points to any AGI-like system having the same epistemological issues as humans, and there's no way around it because of the nature of information.
Stephen Wolfram's computational irreducibility is another one of the issues any self-guided, phyiscally grounded computing engine must have. There are problems that need to be calculated whole. Thinking long and hard about possible end-states won't help. So one would rather have 10000 AGIs doing somewhat similar random search in the hopes that one finds something useful.
I guess this is what we do in global-scale scientific research.
Dead Comment
AIs these days autonomously seek information themselves. Much like living things, they are recycling entropy and information to/from their environment (the internet) at runtime. The framing as a sterile, platonic algorithm is making less and less sense to me with time.
(obviously they differ from living things in lots of other ways, just an example)
I had an experience the other day where claude code wrote a script that shelled out to other LLM providers to obtain some information (unprompted by me). More often it requests information from me directly. My point is that the environment itself for these things is becoming at least as computationally complex or irreducible (as the OP would say) as the model's algorithm, so there's no point trying to analyse these things in isolation.
They're backfeeding what it's "learning" along the way - whether it's in a smart fashion, we don't know yet.
3 Problems with that assumption:
a) Unlike living things, that information doesn't allow them to change. When a human touches a hotplate for the first time, it will (in addition to probably yelling and cursing a lot), learn that hotplates are dangerous and change its internal state to reflect that.
What we currently see as "AI" doesn't do that. Information gathered through means such as websearch + RAG, has ZERO impact on the systems internal makeup.
b) The "AI" doesn't collect the information. The model doesn't collect anything, and in fact can't. It can produce some sequence that may or may not cause some external entity to feed it back some more data (e.g. a websearch, databases, etc.). That is an advantage for technical applications, because it means we can easily marry an LLM to every system imaginable, but its really bad for the prospect of an AGI, that is supposed to be "autonomous".
c) The representation of the information has nothing to do with what it represents. All information an LLM works with, including whatever it is eing fed from th outside, is represented PURELY AND ONLY in terms of statistical relationships between the tokens in the message. There is no world-model, there is no understanding of information. There is mimicry of these things, to the point where they are technically useful and entice humans to anthropomorphise them (a BIIIG chunk of VC money hinges on that), but no actual understanding...and as soon as a model is left to its own devices, which would be a requirement for an AGI (remember: Autonomous), that becomes a problem.
> Unlike living things, that information doesn't allow them to change.
It absolutely does. Their behaviour changes constantly as they explore your codebase, run scripts, question you... this is just plainly obvious to anyone using these things. I agree that somewhere down the line there is a fixed set of tensors but that is not the algorithm. If you want to analyse this stuff in good faith you need to include the rest of the system too, including it's memory, context and more generally any tool it can interact with.
> The "AI" doesn't collect the information.
I really don't know how to engage on this. It certainly isn't me collecting the information. I just tell it what I want it to do at a high level and it goes and does all this stuff on its own.
> There is no world-model, there is no understanding of information.
I'm also not going to engage on this. I could care less what labels people assign to the behaviour of AI agents, and whether it counts as "understanding" or "intelligence" or whatever. I'm interested in their observable behaviour, and how to use them, not so much in the philosophy. In my experience trying to discuss the latter just leads to flame wars (for now).
The paper is talking about whole systems for AGI not the current isolated idea of pure LLM. Systems can store memories without issues. I'm using that for my planning system and the memories and graph triplets get filled out automatically, the get incorporated in future operations.
> It can produce some sequence that may or may not cause some external entity to feed it back some more data
That's exactly what people do while they do research.
> The representation of the information has nothing to do with what it represents.
That whole point implies that the situation is different in our brains. I've not seen anyone describe exactly how our thinking works, so saying this is a limitation for intelligence is not a great point.
Your assertions also make some sense, especially on a technical level. I'd add only that human minds are no longer the only minds utilizing digital tools. There is almost no protective gears or powerful barrier that would likely stand in the way of sentient AIs or AGI trying to "run" and function well on bio cells, like what makes up humans or animals, for the sake of their computational needs and self-interests.
> Anyway, this is not part of the questions this paper seeks to answer. Neither will we wonder in what way it could make sense to measure the strength of a model by its ability to find its relative position to the object it models. Instead, we chose to stay ignorant - or agnostic? - and take this fallible system called "human". As a point of reference.
Cowards.
That's the main counter argument and acknowledging its existence without addressing it is a craven dodge.
Assuming the assumptions[1] are true, then human intelligence isn't even able to be formalized under the same pretext.
Either human intelligence isn't
1. Algorithmic. The main point of contention. If humans aren't algorithmically reducible - even at the level computation of physics, then human cognition is supernatural.
2. Autonomous. Trivially true given that humans are the baseline.
3. Comprehensive (general): Trivially true since humans are the baseline.
4. Competent: Trivially true given humans are the baseline.
I'm not sure how they reconcile this given that they simply dodge the consequences that it implies.
Overall, not a great paper. It's much more likely that their formalism is wrong than their conclusion.
Footnotes
1. not even the consequences, unfortunately for the authors.
–Are we treating an arbitrary ontological assertion as if it’s a formal argument that needs to be heroically refuted? Or better: is that metaphysical setup an argument?
If that’s the game, fine. Here we go:
– The claim that one can build a true, perfectly detailed, exact map of reality is… well... ambitious. It sits remarkably far from anything resembling science , since it’s conveniently untouched by that nitpicky empirical thing called evidence. But sure: freed from falsifiability, it can dream big and give birth to its omnicartographic offspring.
– oh, quick follow-up: does that “perfect map” include itself? If so... say hi to Alan Turing. If not... well, greetings to Herr Goedel.
– Also: if the world only shows itself through perception and cognition, how exactly do you map it “as it truly is”? What are you comparing your map to — other observations? Another map?
– How many properties, relations, transformations, and dimensions does the world have? Over time? Across domains? Under multiple perspectives? Go ahead, I’ll wait... (oh, and: hi too.. you know who)
And btw the true detailed map of the world exists.... It’s the world.
It’s just sort of hard to get a copy of it. Not enough material available ... and/or not enough compute....
P.S. Sorry if that came off sharp — bit of a spur-of-the-moment reply. If you want to actually dig into this seriously, I’d be happy to.
If you are claiming that human intelligence is not "general", you'd better put a huge disclaimer on your text. You are free to redefine words to mean whatever you want, but if you use something so different from the way the entire world uses it, the onus is on you to make it very clear.
And the alternative is you claiming human intelligence is impossible... what would make your paper wrong.
> Are we treating an arbitrary ontological assertion as if it’s a formal argument that needs to be heroically refuted?
The formality of the paper already supposes a level of rigor. The problem at its core, is that p_intelligent(x: X) where X ∈ {human, AI} is not a demonstrable scissor by just proving p_intelligent(AI) = false. Without walking us through the steps that p_intelligent(human) = true, we cannot be sure that the predicate isn't simply always false.
Without demonstrating that humans satisfy the claims we can't be sure if the results are vacuously true because nothing, in fact, can satisfy the standard.
These aren't heroic refutations, they're table stakes.
The rest is just a lot of nit picking and what not for very specific ways to do AGI, very specific definitions of what AGI is, is not, should be, should not be. Etc. Just a lot of people shouting "you're wrong!" at each other for very narrow definitions of what it means to be right. I think that's fundamentally boring.
What it boils down to me is that by figuring out how our own intelligence works, we might stumble upon a path to AGI. And it's not a given that that would be the only path either. At least there appear to be several independently evolved species that exhibit some signs of being intelligent (other than ourselves).
Deleted Comment
https://en.wikipedia.org/wiki/Mind%E2%80%93body_problem
He does touch upon this in section 3, and his argument is - as expected - weak.
Human brains apparently have this set of magic properties that machines can't emulate.
Magical thinking, paper is quackery, don't waste time on it.
> Strange, isn't it? The AI hasn’t crashed. It’s still running.
As a human I answer a question because my time to do so is finite. Why can't we just ask an AI to give its best answer in due time ? As a human I can do that easily. Will my answer be optimal ? No of course, but every manager on earth do that all the time. We're all happy with approximate answers. (and I would add: approximation are sometimes based on our core values, instinct, consciousness, etc.. All things that make us humans, IOW not machines)
"The first difficulty in the way of establishing a probability that one course of action will give a better total result than another, lies in the fact that we have to take account of the effects of both throughout an infinite future. We have no certainty but that, if we do one action now, the Universe will, throughout all time, differ in some way from what it would have been, if we had done another; and, if there is such a permanent difference, it is certainly relevant to our calculation.
But it is quite certain that our causal knowledge is utterly insufficient to tell us what different effects will probably result from two different actions, except within a comparatively short space of time; we can certainly only pretend to calculate the effects of actions within what may be called an ‘immediate’ future. No one, when he proceeds upon what he considers a rational consideration of effects, would guide his choice by any forecast that went beyond a few centuries at most; and, in general, we consider that we have acted rationally, if we think we have secured a balance of good within a few years or months or days."
In a sense, I get why they write verbosely, but...
The first and most important task of our lives is to determine what our goal is.
https://en.wikipedia.org/wiki/Alfred_North_Whitehead#God
Sure you can. One approach is https://arxiv.org/html/2505.11274v2 another is having a parallel "do you want to do more analysis?" agent, and I'm sure someone's already at least experimenting with building the confidence measurement into the layers as well.