All: please don't comment based on your first response to an inevitably shallow title. That leads to generic discussion, which we're trying to avoid on HN. Specific discussion of what's new or different in an article is a much better basis for interesting conversation.
Since we all have language and opinions about it, the risk of genericness is high with a title like this. It's like this with threads about other universal topics too, such as food or health.
The actual paper [1] says that functional MRI (which is measuring which parts of the brain are active by sensing blood flow) indicates that different brain hardware is used for non-language and language functions.
This has been suspected for years, but now there's an experimental result.
What this tells us for AI is that we need something else besides LLMs. It's not clear what that something else is. But, as the paper mentions, the low-end mammals and the corvids lack language but have some substantial problem-solving capability. That's seen down at squirrel and crow size, where the brains are tiny. So if someone figures out to do this, it will probably take less hardware than an LLM.
This is the next big piece we need for AI. No idea how to do this, but it's the right question to work on.
When you look at how humans play chess they employ several different cognitive strategies. Memorization, calculation, strategic thinking, heuristics, and learned experience.
When the first chess engines came out they only employed one of these: calculation. It wasn't until relatively recently that we had computer programs that could perform all of them. But it turns out that if you scale that up with enough compute you can achieve superhuman results with calculation alone.
It's not clear to me that LLMs sufficiently scaled won't achieve superhuman performance on general cognitive tasks even if there are things humans do which they can't.
The other thing I'd point out is that all language is essentially synthetic training data. Humans invented language as a way to transfer their internal thought processes to other humans. It makes sense that the process of thinking and the process of translating those thoughts into and out of language would be distinct.
This seems quite reasonable, but I recently heard a podcast (https://www.preposterousuniverse.com/podcast/2024/06/24/280-...) that LLMs are more likely to be very good at navigating what they have been trained on, but very poor at abstract reasoning and discovering new areas outside of their training. As a single human, you don't notice, as the training material is greater than everything we could ever learn.
After all, that's what Artificial General Intelligence would at least in part be about: finding and proving new math theorems, creating new poetry, making new scientific discoveries, etc.
> It makes sense that the process of thinking and the process of translating those thoughts into and out of language would be distinct
Yes, indeed. And LLMs seem to be very good at _simulating_ the translation of thought into language. They don't actually do it, at least not like humans do.
> It's not clear to me that LLMs sufficiently scaled won't achieve superhuman performance on general cognitive tasks
If "general cognitive tasks" means "I give you a prompt in some form, and you give me an incredible response of some form " (forms may differ or be the same) then it is hard to disagree with you.
But if by "general cognitive task" you mean "all the cognitive things that human do", then it is really hard to see why you would have any confidence that LLMs have any hope of achieving superhuman performance at these things.
> It's not clear to me that LLMs sufficiently scaled won't achieve superhuman performance
To some extent this is true.
To calculate A + B you could for example generate A, B for trillions of combinations and encode that within the network. And it would calculate this faster than any human could.
But that's not intelligence. And Apple's research showed that LLMs are simply inferring relationships based on the tokens it has access to. Which you can throw off by adding useless information or trying to abstract A + B.
Chess is essentially a puzzle. There's a single explicit, quantifiable goal, and a solution either achieves the goal or it doesn't.
Solving puzzles is a specific cognitive task, not a general one.
Language is a continuum, not a puzzle. The problem with LLMs is that testing has been reduced to performance on language puzzles, mostly with hard edges - like bar exams, or letter counting - and they're a small subset of general language use.
Sure, when humans use multiple skill to address a specific problem, you can sometimes outperform them by scaling a spefic one of those skills.
When it comes to general intelligence, I think we are trying to run before we can walk. We can't even make a computer with a basic, animal level understanding of the world. Yet we are trying to take a tool that was developed on top of system that already had an understanding of the world and use it to work backwards to give computers an understanding of the world.
I'm pretty skeptical that we're going to succeed at this. I think you have to be able to teach a computer to climb a tree or hunt (subhuman AGI) before you can create superhuman AGI.
Most pre-deep learning architectures had separate modules like "language model", "knowledge base" and "inference component".
Then LLMs came along, and ML folks got rather too excited that they contain implicit knowledge (which, of course, is required to deal with ambiguity).
Then the new aspiration as "all in one" and "bigger is better", not analyzing what components are needed and how to orchestrate their interplay.
From an engineering (rather than science) point of view, the "end-to-end black box" approach is perhaps misguided, because the result will be a non-transparent system by definition. Individual sub-models should be connected in a way that retains control (e.g. in dialog agents, SRI's Open Agent Architecture was a random example of such "glue" to tie components together, to name but one).
Regarding the science, I do believe language adds to the power of thinking; while (other) animals can of course solve simple problems without language, language permits us to define layers of abstractions (by defining and sharing new concepts) that goes beyond simple, non-linguistic thoughts. Programming languages (created by us humans somewhat in the image of human language) and the language of mathematics are two examples where we push this even further (beyond the definition of new named concepts, to also define new "DSL" syntax) - but all of these could not come into beying without human language: all formal specs and all axioms are ultimately and can only be formulated in human language. So without language, we would likely be stuck at a very simple point of development, individually and collectively.
> I do believe language adds to the power of thinking; while (other) animals can of course solve simple problems without language, language permits us to define layers of abstractions (by defining and sharing new concepts) that goes beyond simple, non-linguistic thoughts.
Based on my experience with toddlers, a rather smart dog, and my own thought processes, I disagree that language is a fundamental component of abstraction. Of sharing abstractions, sure, but not developing them.
When I'm designing a software system I will have a mental conception of the system as layered abstractions before I have a name for any component. I invent names for these components in order to define them in the code or communicate them to other engineers, but the intuition for the abstraction comes first. This is why "naming things" is one of the hard problems in computer science—because the name comes second as a usually-inadequate attempt to capture the abstraction in language.
In my personal learning journey I have been exploring the space of intuitive learning which is dominant in physical skills. Singing requires extremely precise control of actions we can't fully articulate or even rationalise. Teaching those skills requires metaphors and visualising and a whole lot of feedback + trial & error.
I believe that this kind of learning is fundamentally non verbal and we can achieve abstraction of these skills without language. Walking is the most universal of these skills and we learn it before we can speak but if you study it (or better try to program a robot to walk with as many degrees of freedom as the human musculoskeletal system) you will discover that almost all of us don't understand what all the things that go into the "simple" task of walking!
My understanding is that people who are gifted at sports or other physical skills like musical instruments have developed the ability to discover and embed these non verbal abstractions quickly. When I practise the piano and am working on something fast, playing semiquavers at anything above 120bpm is not really conscious anymore in the sense of "press this key then that key"
The concept of arpeggio is verbal but the action is non verbal. In human thought where does verbal and non-verbal start and end? Its probably a continuum
> the "end to end black box" approach is perhaps misguided, because the result will be a non transparent system by definition
A black box that works in human language and can be investigated with perturbations, embedding visualizations and probes. It explains itself as much ore more than we can.
One reason might that LLMs are successful because of the architecture, but also, just as importantly because they can be trained over a volume and diversity of human thought that’s encapsulated in language (that is on the internet). Where are we going to find the equivalent data set that will train this other kind of thinking?
Open AI O1 seems to be trained on mostly synthetic data, but it makes intuitive sense that LLMs work so well because we had the data lying around already.
Language models would seem to be exquisitely tied to the way that evolved intelligence has formulated its society and training.
An Ab Initio AGI would maybe be free of our legacy, but LLMs certainly are not.
I would expect a ship-like intelligence a la the Culture novels to have non-English based cognition. As far as we can tell, our own language generation is post-hoc explanation for thought more so than the embodiment of thought.
Title doesn't mean bullet trains can't fly, but do imply what call flights could be more than moving fast, and effects of wings might be worth discussing.
in the high entropy world we have, we are forced to assume that the first thing that arises as a stable pattern is inevitably the most likely, and the most likely to work. there is no other pragmatic conclusion to draw.
You seem to be conflating "different hardware" with proof that "language hardware" uses "software" equivalent to LLMs.
LLMs basically become practical when you simply scale compute up, and maybe both regions are "general compute", but language ends up on the "GPU" out of pure necessity.
So to me, these are entirely distinct questions: is the language region able to do general cognitive operations? What happens when you need to spell out "ubiquitous" or declense a foreign word in a language with declension (which you don't have memory patterns for)?
I agree it seems obvious that for better efficiency (size of training data, parameter count, compute ability), human brains use different approach than LLMs today (in a sibling comment, I bring up an example of my kids at 2yo having a better grasp of language rules than ChatGPT with 100x more training data).
But let's dive deeper in understanding what each of these regions can do before we decide to compare to or apply stuff from AI/CS.
>What this tells us for AI is that we need something else besides LLMs.
No this is not true. For two reasons.
1. We call these things LLMs and we train it with language but we can also train it with images.
2. We also know LLMs develop a sort of understanding that goes beyond language EVEN when the medium used for training is exclusively language.
The naming of LLMs is throwing you off. You can call it a Large Language Model but this does not mean that everything about LLMs are exclusively tied only to language.
Additionally we don't even know if the LLM is even remotely similar to the way human brains process language.
No such conclusion can be drawn from this experiment.
Brain size isn't necessarily a very good correlate of intelligence. For example dolphins and elephants have bigger brains than humans, and sperm whales have much bigger brains (5x by volume). Neanderthals also had bigger brains than modern humans, but are not thought to have been more intelligent.
A crow has a small brain, but also has very small neurons, so ends up having 1.5B neurons, similar to a dog or some monkeys.
Don’t assume whales are less intelligent than humans. They’re tuned for their environment. They won’t assemble machines with their flippers but let’s toss you naked in the pacific and see if you can communicate and collaborate with peers 200km away on complex hunting strategies.
Dolphins, Orcas, whales and other intelligent cetaceans do not have Hands and live in an environment without access to a technological accelerator like fire.
The absence of both of these things is an incredible crippler for technological development. It doesn't matter how intelligent you are, you're never going to achieve much technologically without these.
I don't think brain size correlations is as straightforward as 'bigger = better' every time but we simply don't know how intelligent most of these species are. Land and Water are completely different beasts.
Right, but what is also important to remember is while size is important what is also key here is the complexity of a neural circuits. Human brain has a lot more connections and is much more complex.
If the suspiciously round number is accurate, this puts the human gut somewhere between a golden hamster and ansell's mole-rat, and about level with a short-palated fruit bat.
I'm not convinced the result is as important here as the methods. Separating language from complex cognition when evaluating individuals is difficult. But many of the people I've met in neuroscience that study language and cognitive processes do not hold the opinion that one is absolutely reliant on the other in all cases. It may have been a strong argument a while ago, but everytime I've seen a presentation on this relationship it's been to emphasize the influence culture and language inevitably have on how we think about things. I'm sure some people believe that one cannot have complex thoughts without language, but most people in speech neuro I've met in language processing research find the idea ridiculous enough they wouldn't bother spending a few years on that kind of project just to disapprove a theory.
On the other hand, further understanding how to engage complex cognitive processes in nonverbal individuals is extremely useful and difficult to accomplish.
Is it important? To who? Anyone with half a brain is aware that language isn't the only way to think. I can think my way through all kinds of things in 3-d space without a single word uttered in any internal monologue and I'm not remotely unique - this kind of thing is put in all kinds of math and iq'ish like tests one takes as a child.
Before you say things this patiently dumb you should probably wonder what question the researchers are actually interested in and why your average experience isn't sufficient proof.
You highlight an expectation that the “truer intelligence” is a singular device, once isolated would mobilize ultimate AGI.
All intelligence is the mitigation of uncertainty (the potential distributed problem.) if it does not mitigate uncertainty it is not intelligence, it is something else.
Intelligence is a technology. For all life intelligence and the infrastructure of performing work efficiently (that whole entropy thing again) is a technology. Life is an arms race to maintain continuity (identity, and the very capacity of existential being.)
The modern problem is achieving reliable behavioral intelligence (constrained to a specific problem domain.) AGI is a phantasm. What manifestation of intelligence appears whole and complete and is always right? These are the sorts of lies you tell yourself, the ones that get you into trouble. They distract from tangible real world problems, perhaps causing some of them. True intelligence is a well calibrated “scalar” domain specific problem (uncertainty) reducer. There are few pressing idempotent obstructions in the real world.
Intelligence is the mitigation of uncertainty.
Uncertainty is the domain of negative potential (what,where,why,how?)
Mitigation is the determinant resolve of any constructive or destructive interference affecting (terminal resolve within) the problem domain.
Examples of this may be piled together mountains high, and you may call that functional AGI, though you would be self deceiving. At some point “good enough” may be declared for anything so passing as yourselves.
LLM as a term is becoming quite broad; a multi-modal transformer-based model with function calling / ReAct finetuning still gets called an LLM, but this scaffolding might be all that’s needed.
I’d be extremely surprised if AI recapitulates the same developmental path as humans did; evolution vs. next-token prediction on an existing corpus are completely different objective functions and loss landscapes.
I asked both OpenAI and Claude the same difficult programming question. Each gave a nearly identical response down to the variable names and example values.
I then looked it up and they had each copy/pasted the same Stack overflow answer.
Furthermore, the answer was extremely wrong, the language I used was superficially similar to the source material, but the programming concepts were entirely different.
What this tells me is there is clearly no “reasoning” happening whatsoever with either model, despite marketing claiming as such.
> Recent work has revealed that the neural activity patterns correlated with sensation, cognition, and action often are not stable and instead undergo large scale changes over days and weeks—a phenomenon called representational drift.
[...]
So, I'm not sure how conclusive this fmri activation study is either.
Though, is there a proto language that's not even necessary for the given measured aspects of condition?
Which artificial network architecture best approximates which functionally specialized biological neutral networks?
For those interested in the history, this is in fact the Neural Network research path that predated LLMs. Not just in the sense that Hinton et al and the core of the "Parallel Distributed Processing"/Connectionist school were always opposed to Chomsky's identification of brain-thought-language, but that the original early 2000s NSF grant awarded to Werbos, Ng, LeCun et al was for "Deep Learning in the Mammalian Visual Cortex." In their research program, mouse intelligence was posited as the first major challenge.
Not sure about that. The same abstract model could be used for both (symbols generated in sequence). For language the symbols have meaning in the context of language. For non-language thought they don't. Nature seems to work this way in general: re-using/purposing the same underlying mechanism over and over at different levels in the stack. All of this could be a fancy version of very old hardware that had the purpose of controlling swimming direction in fish. Each symbol is a flick of the tail.
I like to think of the non-verbal portions as the biological equivalents of ASICs. even skills like riding a bicycle might start out as conscious effort (a vision model, a verbal intention to ride and a reinforcement learning teacher) but is then replaced by a trained model to do the job without needing the careful intentional planning. some of the skills in the bag of tricks are fine tuned by evolution.
ultimately, there's no reason that a general algorithm couldn't do the job of a specific one, just less efficiently.
> What this tells us for AI is that we need something else besides LLMs.
An easy conclusion to jump to but I believe we need to be more careful. Nothing in these findings proves conclusively that non-verbal reasoning mechanism equivalent to humans couldn't evolve in some part of a sufficiently large ANN trained on text and math. Even though verbal and non-verbal reasoning occurs in two distinct parts of the brain, it doesn't mean they're not related.
Saussure already pointed out these issues over a century ago, and Linguists turned ML Researchers like Stuart Russell and Paul Smolensky tried in vain to resolve this.
It basically took 60 years just to crack syntax at scale, and the other layers are still fairly far away.
Furthermore, Syntax is not a solved problem yet in most languages.
Try communicating with GPT-4o in colloquial Bhojpuri, Koshur, or Dogri, let alone much less represented languages and dialects.
Higher order faculties aside, animals seem like us, just simpler.
The higher functioning ones appear to have this missing thing too. We can see it in action. Perhaps all of them do and it is just harder for us when the animal thinks very differently or maybe does not think as much, feeling more, for example.
----
Now, about that thing... and the controversy:
Given an organism, or machine for this discussion, is of sufficiently robust design and complexity that it can precisely differentiate itself from everything else, it is a being.
This thing we are missing is an emergent property, or artifact that can or maybe always does present when a state of being also presents.
We have not created a machine of this degree yet.
Mother nature has.
The reason for emergence is a being can differentiate sensory input as being from within, such as pain, or touch, and from without, such as light or motion.
Another way to express this is closed loop vs open loop.
A being is a closed loop system. It can experience cause and effect. It can be the cause. It can be the effect.
A lot comes from this closed loop.
There can be the concept of the self and it has real meaning due to the being knowing what is of itself or something, everything else.
This may be what forms consciousness. Consciousness may require a closed loop, and organism of sufficient complexity to be able to perceive itself.
That is the gist of it.
These systems we make are fantastic pieces. They can pattern match and identify relationships between the data given in amazing ways.
But they are open loop. They are not beings. They cannot determine what is part of them, what they even are,or anything really.
I am both consistently amazed and dismayed at what we can get LLM systems to do.
They are tantalizingly close!
We found a piece of how all this works and we are exploiting the cral out of it. Ok fine. Humans are really good at that.
But it will all taper off. There are real limits because we will eventually find the end goal will be to map out the whole problem space.
Who has tried computing that? It is basically all possible human thought. Not going to happen.
More is needed.
And that "more" can arrive at thoughts without having first seen a few bazillion to choose from.
The projects mapping the brain, combined with research on what areas do, should tell us what components are necessary for our design. Studying the behavior of their specialist structures will tell us how to make purpose-built components for these tasks. Even if not, just attempting to split up the global behavior in that many ways with specialist architecture might help. We can also imitate how the components synchronize together, too.
An example was the problem of memory shared between systems. ML people started doing LLM’s with RAG. I looked into neuroscience which suggested we need a hippocampus model. I found several papers with hippocampus-like models. Combining LLM’s, vision, etc with hippocampus-like model might get better results. Rinse repeat for these other brain areas wherever we can understand them.
I also agree on testing the architectures with small, animal brains. Many do impressive behaviors that we should be able to recreate in simulators or with robotics. Some are useful, too, like how geese are good at security. Maybe embed a trained, goose brain into a camera system.
> What this tells us for AI is that we need something else besides LLMs
I am not convinced it follows. Sure LLMs don’t seem complete however there’s a lot of unspoken inference going on in LLMs that don’t map into a language directly already - the inner layers of the deep neural net that operates on abstract neurons.
> What this tells us for AI is that we need something else besides LLMs.
Perhaps, but the relative success of trained LLMs acting with apparent generalised understanding may indicate that it is simply the interface that is really an LLM post training;
That the deeper into the network you go (the further from the linguistic context), the less things become about words and linguist structure specifically and the more it becomes about things and relations in general.
(This also means that multiple interfaces can be integrated, sometimes making translation possible, e.g.: image <=> tree<string>)
My first thought as well - “AGI via LLM” implies that our grey matter is merely a substrate for executing language tasks: just swap out bio-neurons for a few H100s and viola, super intelligence.
> So if someone figures out to do this, it will probably take less hardware than an LLM.
We have, it's called DreamCoder. There's a paper and everything.
Everything needed for AGI exists today, people simply have (incorrect) legacy beliefs about cognition that are holding them back (e.g. "humans are rational").
> What this tells us for AI is that we need something else besides LLMs.
Despite being an LLM skeptic of sorts, I don’t think that necessarily follows. The LLM matrix multiplication machinery may well be implementing an equivalent of the human non-language cognitive processing as a side effect of the training. Meaning, what is separated in the human brain may be mixed together in an LLM.
You are getting derailed because of the name we've chosen to call these models but only the first few and last few layers of LLMs deal with tokens. The rest deal with abstract representations and models learnt during training. Language goes in and Language comes out but Language is not the in-between for either LLMs or Humans.
I'm curious why "simulation" isn't the extra thing needed? Yes, we need language to communicate ideas. But you can simulate in your mind things happening that you don't necessarily have words for, yet. Right?
Interestingly though for AI, this doesn’t necessarily mean we need a different model architecture. A single large multimodal transformer might be capable of a lot that an LLM is not (besides the multimodality).
We need to add the 5 senses, of which we have now image, audio and video understanding in LLMs. And for agentic behavior they need environments and social exposure.
This is actually exactly what is needed. We think the dataset is the primary limitation to an LLMs capability but in reality we are only developing one part of their "intelligence" - a functional and massive model isn't the end of their training - its kinda just the beginning.
Not only that but also LLMs "think" in a latent representation that is several layers deep. Sure, the first and last layers make it look like it is doing token wrangling, but what is happening in the middle layers is mostly a mystery. First layer deals directly with the tokens because that is the data we are observing (a "shadow" of the world) and last layer also deals with tokens because we want to understand what the network is "thinking" so it is a human specific lossy decoder (we can and do remove that translator and plug the latent representations to other networks to train them in tandem). There is no reason to believe that the other layers are "thinking in language".
At times I had impaired brain function (lots of soft neurological issues, finger control, memory loss, balance issues) but surprisingly the core area responsible for mathematical reasoning was spared .. that was a strange sensation, almost schizophrenic.
And yeah it seems that core primitives of intelligence exist very low in our brains. And with people like Michael Levin, there may even be a root beside nervous systems.
It’s impossible to overstate how crude and useless blood flow MRI studies are, at least relative to the hype they receive.
Spoiler alert: brains require a lot of blood, constantly, just to not die. Looking at blood flow on an MRI to determine neural circuitry has to deal with the double whammy of both an extremely crude tool and a correlation/causation fallacy.
As some who has a dis-harmonic intelligence profile, this has been obvious for a very long time. In the family of my mother there are several individuals struggling with language while excelling in the field of exact sciences. I very strongly suspect that my non-verbal (performal) IQ is much higher (around 130) than my verbal IQ (around 100). I have struggled my whole life to express my ideas with language. I consider myself an abstract visual thinker. I do not think in pictures, but in abstract structures. During my life, I have met several people, especially among software engineers, who seem to be similar to me. I also feel that people who are strong verbal thinkers have the greatest resistance against idea that language is not essential for higher cognitive processes.
> As some who has a dis-harmonic intelligence profile, this has been obvious for a very long time. In the family of my mother there are several individuals struggling with language while excelling in the field of exact sciences. I very strongly suspect that my non-verbal (performal) IQ is much higher (around 130) than my verbal IQ (around 100)
I used to rationalize to myself along similar lines for a long time, then I realized that I'm just not as smart as I thought I was.
That was a difficult thing for me as well -- if you have such great ideas in your head but they fall apart once you try to bring them down on paper, maybe those ideas simply aren't that great.
I'm a brilliant genius according to IQ tests. Think me arrogant or conceited or whatever - that is literally the truth, fact - proven many times in the educational system (I was homeschooled and didn't follow any sort of curriculum and was allowed to do whatever I wanted bc I kept testing higher than almost everyone) and just for kicks also - the last time I took an IQ test I was in my late 20s and a friend and I had a bet about who could score higher completely stoned off of our ass. We rolled enough blunts apiece that we could be continuously smoking marijuana as we took the IQ test, which followed several bongs finished between the two of us. I was so high that I couldn't keep the numbers straight on one of the number pattern questions - it was ridiculous. I scored 124, my lowest "serious" attempt ever - all of this is 100% true. I need anyone to believe me - take this how you will but I have an opinion that is a bit different.
I'm brilliant - I've read volumes of encyclopedias, my hobbies include comparative theology, etymology, quantum mechanics and predicting the future with high accuracy (I only mention stuff I'm certain of tho ;) but so much so it disturbs my friends and family.
The highest I scored was in the 160s as a teenager but I truly believe they were over compensating for my age - only as an adult have I learned most children are stupid and they maybe in fact didn't over compensate. I am different than anyone else I've ever personally met - I fundamentally see the world different.
All of that is true but that's a rather flawed way of assessing intelligence - fr. I'm being serious. The things we know can free us as much as they can trap us - knowledge alone doesn't make a man successful, wealthy, happy or even healthy - I'm living evidence of this. That doesn't cut it as a metric for prediction of much. There are other qualities that are far more valuable in the societal sense.
Every Boss I've ever worked for has been dumber than me - each one I've learned invaluable stuff from. I was a boss once - in my day I owned and self taught/created an entire social network much like FB was a few years ago, mine obviously didn't take off and now I'm a very capable bum. Maybe someday something I'm tinkering with will make me millions but prolly not, for many reasons, I could write books if I wanted ;)
At the end of the day, the facts are what they are - there is an optimal level of intelligence that is obviously higher than the bottom but is nowhere near the top tier, very likely near that 100 IQ baseline. What separates us all is our capabilities - mostly stuff we can directly control, like learning a trade.
A Master Plumber is a genius plumber by another name and that can and obviously is most often, learned genius. What you sus about yourself is truth - don't doubt that. No IQ test ever told me I lacked the tenacity of the C average student that would employ me someday - they can't actually measure the extent of our dedicated capacity.
I kno more than most people ever have before or rn presently - I don't know as much about plumbing as an apprentice with 2 years of a trade school dedicated to plumbing and a year or two of experience in the field, that's the reality of it. I could learn the trade - I could learn most every trade, but I won't. That's life. I can tell you how you the ancients plumbed bc that piqued my curiosity and I kno far more about Roman plumbing than I do how a modern city sewer system works. That's also life.
It isn't what we kno or how fast we can learn it - it's what we do that defines us.
Become more capable if you feel looked down on - this is the way bc even if what you hone your capabilities of can be replicated by others most won't even try.
That's my rant about this whole intelligence perception we currently have as a society. Having 100 IQ is nowhere near the barrier that having 150 IQ is.
Rant aside, to the article - how isn't this obvious? I mean feelings literally exist - not just the warm fuzzy ones, like the literal feeling of existence. Does a monkey's mind require words to interpret pain or pleasure for example. Do I need to know what "fire" or "hot" is in a verbal context to sufficiently understand "burn" - words exists to convey to to others what doesn't need to be conveyed to us. That's their function. Communication. To facilitate communication with our social brethren we adopt them fundamentally as our Lego blocks for understanding the world - we pretend that words comprising language are the ideas themselves. A banana is a - the word is the fruit, they are the same in our minds but if I erase the word banana and all it's meaning of the fruit and I randomly encounter a banana - I still can taste it. No words necessary.
Also, you can think without words, deliberately and consciously - even absentmindedly.
And LLMs can't reason ;)
Truthfully, the reality is that a 100 IQ normal human is far more capable than any AI I've been given access to - in almost every metric I attempted to asses I ultimately didn't even bother as it was so obvious that humans are functionally superior.
When AI can reason - you, and everyone else, will kno it. It will be self evident.
Anyways, tldr: ppl are smarter than given credit for, smarter and much more capable - IQ is real and matters but far less than we are led to believe. People are awesome - the epitome of biological life on Earth and we do a lot of amazing things and anyone can be amazing.
I hate it when the Hacker News collective belittles itself - don't do that. I rant here bc it's one of the most interesting places I've found and I care about what all of you think far more than I care about your IQ scores.
Growing up, I never used words or even sentences for thinking.
The abstract visualizations I could build in my mind where comparable to semi-transparent buildings that I could freely spin, navigate and bend to connect relations.
In my mid-twenties, someone introduced me to the concept of people using words for mental processes, which was completely foreign to me up to this point.
For some reason, this made my brain move more and more towards this language-based model and at the same time, I felt like I was losing the capacity for complex abstract thoughts.
Still to this day I (unsuccessfully) try to revive this and unlearn the language in my head, which feels like it imposes a huge barrier and limits my mental capacity to the capabilities of what the language my brain uses at the given time (mostly EN, partially DE) allows to express.
This reminds me of my experiences working with a software developer transplanted from the humanities who was highly articulate and capable of producing language about programming, yet seemed to not be able to write many actual computer programs themselves.
I think that I ultimately developed an obsessive need to cite all my ideas against the literature and formulate natural language arguments for my claims to avoid being bludgeoned over the head with wordcelry and being seen as inferior for my lesser verbal fluency despite having written software for years at that point, since early childhood, and even studied computer science.
I think people who can manipulate complex structures but struggle with language tend to see language in a more formal way, putting more effort into understanding its structure and inner working.
Basically what to most people is so obvious that it becomes transparent ("air") isn't to us, which apparently is an incredible gift for becoming a language researcher. Or a programmer.
Although I fit the profile of a verbal thinker (English degree, education in the humanities) I don't exactly find language the primary aspect of my thought.
It seems more like a complement to it: the idea arises, and then I have this compulsion to verbalise it, which gets quite frustrating as it takes several iterations. Clearly words do matter to me as a way to structure and record my ideas but there is something that pre-empts verbalisation and to some extent resists it.
I cannot provide insight on how I arrive at ideas. Even when I did literary criticism, the best I can say is that I absorbed lots of text and then suddenly a pattern would spring out. But the same things would happen for me studying maths or the hard sciences.
Software engineering is actually a bit different for me because I am not naturally a good algorithmic problem solver. Really I am somebody very passionate about computing who has a near-compulsion to see and collect more and more technology. So for me it is as simple as saying "this resembles a reader monad" or "this puns on the active record pattern". Less impressive than my humanities intelligence but worth maybe 10x the amount in the labour market :-)
> During my life, I have met several people, especially among software engineers, who seem to be similar to me
This begs a question though: Since programming is mostly done with language - admittedly primitive/pidgin ones - why isn't that a struggle? Not sure if you're a programmer yourself, but if so do you prefer certain programming languages for some sense of "less-verbalness" or does it even matter?
The idea that programming languages and natural languages are processed with the same wetware should be testable with something like the tests described in this submission.
I don't expect it to be true, but only expecting something is not science
I see your general point on needing language proficiency to program, but I think it's just a very low requirement.
Parent isn't saying they can't handle language (and we wouldn't have this discussion in the first place), just that they better handle complexity and structure in non verbal ways.
To get back to programming, I think this do apply to most of us. Most of us probably don't think in ruby or JS, we have a higher vision of what we want to build and "flatten" it into words that can be parsed and executed. It's of course more obvious for people writing in say basic or assembly, some conversion has to happen at some point.
Programming is moreso based on recursive problem solving. (Most) language does have some recursive structures, but these become quite difficult to think about after just a few levels, and really aren't what you'd normally consider to be "good language", e.g.
> The dog's owner's house's roof's angle's similarity to an equilateral triangle is remarkable.
You’d still be reasoning using symbols, language is inherently an extension of symbols and memes. Think of a person representing a complex concept in their mind with a symbol and using it for further reasoning
>> They’re basically the first model organism for researchers studying the neuroscience of language. They are not a biological organism, but until these models came about, we just didn’t have anything other than the human brain that does language.
I think this is completely wrong-headed. It's like saying that until cars came about we just didn't have anything other than animals that could move around under its own power, therefore in order to understand how animals move around we should go and study cars. There is a great gulf of unsubstantiated assumptions between observing the behaviour of a technological artifact, like a car or a statistical language model, and thinking we can learn something useful from it about human or more generally animal faculties.
I am really taken aback that this is a serious suggestion: study large language models as in-silico models of human linguistic ability. Just putting it down in writing like that rings alarm bells all over the place.
I've been trying to figure out to respond to this for a while. I appreciate the fact that you are pretty much the lone voice on this thread voicing this opinion, which I also share but tend to keep my mouth shut since it seems to be unpopular.
It's hard for me to understand where my peers are coming from on the other side of this argument
and respond without being dismissive, so I'll
do my best to steelman the argument later.
Machine learning models are function approximators and by definition do not have an internal experience distinct from the training data any more
than the plus operator does. I agree with the sentiment that even putting it in writing gives more weight to the position than it should, bordering on absurdity.
I suppose this is like the ELIZA phenomena on steroids, is the only thing I can think of for why such notions are being entertained.
However, to be generous, lets do some vigorous hand waving and say we could find a way to have an embodied learning agent gather sublinguistic perceptual data in an online reinforcement learning process, and furthermore that the (by definition) non-quantifiable subjective experience data could somehow be extracted, made into a training set, and fit to a nicely parametric loss function.
The idea then is that could find some architecture that would allow you to fit a model to the data.
And voila, machine consciousness, right? A perfect model for sentience.
Except for the fact that you would need to ignore that in the RL model gathering the data and the NN distilled from it, even with all of our vigorous hand waving, you are once again developing function approximators that have no subjective internal experience distinct from the training data.
Let's take it one step further. The absolute simplest form of learning comes in the form of habituation and sensitization to stimuli. Even microbes have the ability to do this.
LLMs and other static networks do not. You can attempt to attack this point by fiatting online reinforcement learning or dismissing it as unnecessary, but I should again point out that you would be attacking or dismissing the bare minimum requirement for learning, let alone a higher order subjective internal experience.
So then the argument, proceeding from false premises, would claim that the compressed experience in the NN could contain mechanical equivalents of higher order internal subjective experiences.
So even with all the might vigorous hand waving we have allowed, you have at best found a way to convert internal subjective processes to external mechanical processes fit to a dataset.
The argument would then follow, well, what's the difference? And I could point back to the microbe, but if the argument hasn't connected by this point, we will be chasing our tails forever.
A good book on the topic that examines this in much greater depth is "The Self Assembling Brain".
I could enter what we all here call the "Zone" quite often when i was young (once while doing math :D). I still can, but rarely on purpose, and rarely while coding. I have a lot of experience in this state, and i can clearly say that a marker of entering the zone is that your thoughts are not "limited" by language anymore and the impression of clarity and really fast thinking. This is why i never thought that language was required for thinking.
Now the question: would it be possible to scan the brain of people while they enter the zone? I know it isn't a state you can reach on command, but isn't it worth to try? understand the mechanism of this state? And maybe understand where our thought start?
I also wonder - is the flow state for work the same as the flow state in other domains? IE team sports flow state is very similar - actions flow smoothly and feel automatic. Flow state in cycling feels really similar, but doesn’t create the same outputs.
I’m not a neuroscience expert, but I do have a degree in philosophy. The Russell quote immediately struck me as misleading (especially without a citation). The author could show more integrity by including Russell’s full quote:
> Language serves not only to express thoughts, but to make possible thoughts which could not exist without it. It is sometimes maintained that there can be no thought without language, but to this view I cannot assent: I hold that there can be thought, and even true and false belief, without language. But however that may be, it cannot be denied that all fairly elaborate thoughts require words.
> Human Knowledge: Its Scope and Limits by Bertrand Russell, Section: Part II: Language, Chapter I: The Uses of Language Quote Page 60, Simon and Schuster, New York.
Of course, that would contravene the popular narrative that philosophers are pompous idiots incapable of subtlety.
Is Russell aligned with Ludwig Wittgenstein’s statement, "The limits of my language mean the limits of my world."? Is he talking about how to communicate his world to others, or is he saying that without language internal reasoning is impossible?
Practically, I think the origins of fire-making abilities in humans tend to undermine that viewpoint. No other species is capable of reliably starting a fire with a few simple tools, yet the earliest archaeological evidence for fire (1 mya) could mean the ability predated complex linguistic capabilities. Observation and imitation could be enough for transmitting the skill from the first proto-human who successfully accomplished the task to others.
P.S. This is also why Homo sapiens should be renamed Homo ignis IMO.
I think it’s a nicely summarised challenge to boot.
It’s doubtless to me that thinking happens without intermediary symbols; but it’s also obvious that I can’t think deeply without the waypoints and context symbols provide. I think it is a common sense opinion.
"Language" is a subset of "symbols". I agree with what you said, but it's not representative of the quote in GP.
Just a few days ago was "What do you visualize while programming?", and there's a few of us in the comments that, when programming, think symbolically without language: https://news.ycombinator.com/item?id=41869237
The important question is: what is considered a language?
> You can ask whether people who have these severe language impairments can perform tasks that require thinking. You can ask them to solve some math problems or to perform a social reasoning test, and all of the instructions, of course, have to be nonverbal because they can’t understand linguistic information anymore.
Surely these "non-verbal instructions" are some kind of language. Maybe all human action can be considered language.
A contrarian example to this research might be feral children, i.e people who have been raised away from humans.[0] In most cases they are mentally impaired; as in not having human-like intelligence. I don't think there is a good explanation why this happens to humans. And why it doesn't happen to other animals, which develop normally in species-typical way whether they are in the wild or in human captivity. It seems that most human behavior (even high-level intelligence) is learned / copied from other humans, and maybe this copied behavior can be considered language.
If humans are "copy machines", there's also a risk of completely losing the "what's it like to be a human" behavior if children of the future are raised by AI and algorithmic feeds.
It's worth noting the precise and narrow sense in which the term "language" is used throughout these studies: it is those particular "word sequences" that activate particular regions in the brain's left hemisphere, to the exclusion of other forms of symbolic representation such as mathematical notation. Indeed, in two of the studies cited, [0] [1] subjects with language deficits or brain lesions in areas associated with the "language network" are asked to perform on various mathematical tasks involving algebraic expressions [0] or Arabic numerals [1]:
> DA was impaired in solving simple addition, subtraction, division or multiplication problems, but could correctly simplify abstract expressions such as (b×a)÷(a×b) or (a+b)+(b+a) and make correct judgements whether abstract algebraic equations like b − a = a − b or (d÷c)+a=(d+a)÷(c+a) were true or false.
> Sensitivity to the structural properties of numerical expressions was also evaluated with bracket problems, some requiring the computation of a set of expressions with embedded brackets: for example, 90 [(3 17) 3].
Discussions of whether or not these sorts of algebraic or numerical expressions constitute a "language of mathematics" aside (despite them not engaging the same brain regions and structures associated with the word "language"); it may be the case that these sorts of word sequences and symbols processed by structures in the brain's left hemisphere are not essential for thought, but can still serve as a useful psychotechnology or "bicycle of the mind" to accelerate and leverage its innate capabilities. In a similar fashion to how this sort of mathematical notation allows for more concise and precise expression of mathematical objects (contrast "the number that is thrice of three and seventeen less of ninety") and serves to amplify our mathematical capacities, language can perhaps be seen as a force multiplier; I have doubts whether those suffering from aphasia or an agrammatic condition would be able to rise to the heights of cognitive performance.
Readit News
Since we all have language and opinions about it, the risk of genericness is high with a title like this. It's like this with threads about other universal topics too, such as food or health.
The actual paper [1] says that functional MRI (which is measuring which parts of the brain are active by sensing blood flow) indicates that different brain hardware is used for non-language and language functions. This has been suspected for years, but now there's an experimental result.
What this tells us for AI is that we need something else besides LLMs. It's not clear what that something else is. But, as the paper mentions, the low-end mammals and the corvids lack language but have some substantial problem-solving capability. That's seen down at squirrel and crow size, where the brains are tiny. So if someone figures out to do this, it will probably take less hardware than an LLM.
This is the next big piece we need for AI. No idea how to do this, but it's the right question to work on.
[1] https://www.nature.com/articles/s41586-024-07522-w.epdf?shar...
When the first chess engines came out they only employed one of these: calculation. It wasn't until relatively recently that we had computer programs that could perform all of them. But it turns out that if you scale that up with enough compute you can achieve superhuman results with calculation alone.
It's not clear to me that LLMs sufficiently scaled won't achieve superhuman performance on general cognitive tasks even if there are things humans do which they can't.
The other thing I'd point out is that all language is essentially synthetic training data. Humans invented language as a way to transfer their internal thought processes to other humans. It makes sense that the process of thinking and the process of translating those thoughts into and out of language would be distinct.
After all, that's what Artificial General Intelligence would at least in part be about: finding and proving new math theorems, creating new poetry, making new scientific discoveries, etc.
There is even a new challenge that's been proposed: https://arcprize.org/blog/launch
> It makes sense that the process of thinking and the process of translating those thoughts into and out of language would be distinct
Yes, indeed. And LLMs seem to be very good at _simulating_ the translation of thought into language. They don't actually do it, at least not like humans do.
If "general cognitive tasks" means "I give you a prompt in some form, and you give me an incredible response of some form " (forms may differ or be the same) then it is hard to disagree with you.
But if by "general cognitive task" you mean "all the cognitive things that human do", then it is really hard to see why you would have any confidence that LLMs have any hope of achieving superhuman performance at these things.
To some extent this is true.
To calculate A + B you could for example generate A, B for trillions of combinations and encode that within the network. And it would calculate this faster than any human could.
But that's not intelligence. And Apple's research showed that LLMs are simply inferring relationships based on the tokens it has access to. Which you can throw off by adding useless information or trying to abstract A + B.
Solving puzzles is a specific cognitive task, not a general one.
Language is a continuum, not a puzzle. The problem with LLMs is that testing has been reduced to performance on language puzzles, mostly with hard edges - like bar exams, or letter counting - and they're a small subset of general language use.
When it comes to general intelligence, I think we are trying to run before we can walk. We can't even make a computer with a basic, animal level understanding of the world. Yet we are trying to take a tool that was developed on top of system that already had an understanding of the world and use it to work backwards to give computers an understanding of the world.
I'm pretty skeptical that we're going to succeed at this. I think you have to be able to teach a computer to climb a tree or hunt (subhuman AGI) before you can create superhuman AGI.
https://arstechnica.com/ai/2024/10/llms-cant-perform-genuine...
or do you maybe think no logical reasoning is needed to do everything a human can do? Tho humans seem to be able to do logical reasoning
Then LLMs came along, and ML folks got rather too excited that they contain implicit knowledge (which, of course, is required to deal with ambiguity). Then the new aspiration as "all in one" and "bigger is better", not analyzing what components are needed and how to orchestrate their interplay.
From an engineering (rather than science) point of view, the "end-to-end black box" approach is perhaps misguided, because the result will be a non-transparent system by definition. Individual sub-models should be connected in a way that retains control (e.g. in dialog agents, SRI's Open Agent Architecture was a random example of such "glue" to tie components together, to name but one).
Regarding the science, I do believe language adds to the power of thinking; while (other) animals can of course solve simple problems without language, language permits us to define layers of abstractions (by defining and sharing new concepts) that goes beyond simple, non-linguistic thoughts. Programming languages (created by us humans somewhat in the image of human language) and the language of mathematics are two examples where we push this even further (beyond the definition of new named concepts, to also define new "DSL" syntax) - but all of these could not come into beying without human language: all formal specs and all axioms are ultimately and can only be formulated in human language. So without language, we would likely be stuck at a very simple point of development, individually and collectively.
EDIT: 2 typos fixed
Based on my experience with toddlers, a rather smart dog, and my own thought processes, I disagree that language is a fundamental component of abstraction. Of sharing abstractions, sure, but not developing them.
When I'm designing a software system I will have a mental conception of the system as layered abstractions before I have a name for any component. I invent names for these components in order to define them in the code or communicate them to other engineers, but the intuition for the abstraction comes first. This is why "naming things" is one of the hard problems in computer science—because the name comes second as a usually-inadequate attempt to capture the abstraction in language.
In my personal learning journey I have been exploring the space of intuitive learning which is dominant in physical skills. Singing requires extremely precise control of actions we can't fully articulate or even rationalise. Teaching those skills requires metaphors and visualising and a whole lot of feedback + trial & error.
I believe that this kind of learning is fundamentally non verbal and we can achieve abstraction of these skills without language. Walking is the most universal of these skills and we learn it before we can speak but if you study it (or better try to program a robot to walk with as many degrees of freedom as the human musculoskeletal system) you will discover that almost all of us don't understand what all the things that go into the "simple" task of walking!
My understanding is that people who are gifted at sports or other physical skills like musical instruments have developed the ability to discover and embed these non verbal abstractions quickly. When I practise the piano and am working on something fast, playing semiquavers at anything above 120bpm is not really conscious anymore in the sense of "press this key then that key"
The concept of arpeggio is verbal but the action is non verbal. In human thought where does verbal and non-verbal start and end? Its probably a continuum
A black box that works in human language and can be investigated with perturbations, embedding visualizations and probes. It explains itself as much ore more than we can.
Not to over-hype LLMs, but I don't see why this results says this. AI doesn't need to do things the same way as evolved intelligence has.
Open AI O1 seems to be trained on mostly synthetic data, but it makes intuitive sense that LLMs work so well because we had the data lying around already.
Similar reason we look for markers of Earth-based life on alien planets: it's the only example we've got of it existing.
An Ab Initio AGI would maybe be free of our legacy, but LLMs certainly are not.
I would expect a ship-like intelligence a la the Culture novels to have non-English based cognition. As far as we can tell, our own language generation is post-hoc explanation for thought more so than the embodiment of thought.
for more, see "Assembly Theory"
LLMs basically become practical when you simply scale compute up, and maybe both regions are "general compute", but language ends up on the "GPU" out of pure necessity.
So to me, these are entirely distinct questions: is the language region able to do general cognitive operations? What happens when you need to spell out "ubiquitous" or declense a foreign word in a language with declension (which you don't have memory patterns for)?
I agree it seems obvious that for better efficiency (size of training data, parameter count, compute ability), human brains use different approach than LLMs today (in a sibling comment, I bring up an example of my kids at 2yo having a better grasp of language rules than ChatGPT with 100x more training data).
But let's dive deeper in understanding what each of these regions can do before we decide to compare to or apply stuff from AI/CS.
No this is not true. For two reasons.
1. We call these things LLMs and we train it with language but we can also train it with images.
2. We also know LLMs develop a sort of understanding that goes beyond language EVEN when the medium used for training is exclusively language.
The naming of LLMs is throwing you off. You can call it a Large Language Model but this does not mean that everything about LLMs are exclusively tied only to language.
Additionally we don't even know if the LLM is even remotely similar to the way human brains process language.
No such conclusion can be drawn from this experiment.
A crow has a small brain, but also has very small neurons, so ends up having 1.5B neurons, similar to a dog or some monkeys.
The absence of both of these things is an incredible crippler for technological development. It doesn't matter how intelligent you are, you're never going to achieve much technologically without these.
I don't think brain size correlations is as straightforward as 'bigger = better' every time but we simply don't know how intelligent most of these species are. Land and Water are completely different beasts.
And it turns out that human brain volume and intelligence are moderately-highly correlated [1][2]!
[1]: https://pmc.ncbi.nlm.nih.gov/articles/PMC7440690/ [2]: https://www.sciencedirect.com/science/article/abs/pii/S01602...
https://www.scientificamerican.com/article/gut-second-brain/
There are 100 million in my gut, but it doesn't solve any problems that aren't about poop, as far as I know.
https://en.wikipedia.org/wiki/List_of_animals_by_number_of_n...
If the suspiciously round number is accurate, this puts the human gut somewhere between a golden hamster and ansell's mole-rat, and about level with a short-palated fruit bat.
On the other hand, further understanding how to engage complex cognitive processes in nonverbal individuals is extremely useful and difficult to accomplish.
All intelligence is the mitigation of uncertainty (the potential distributed problem.) if it does not mitigate uncertainty it is not intelligence, it is something else.
Intelligence is a technology. For all life intelligence and the infrastructure of performing work efficiently (that whole entropy thing again) is a technology. Life is an arms race to maintain continuity (identity, and the very capacity of existential being.)
The modern problem is achieving reliable behavioral intelligence (constrained to a specific problem domain.) AGI is a phantasm. What manifestation of intelligence appears whole and complete and is always right? These are the sorts of lies you tell yourself, the ones that get you into trouble. They distract from tangible real world problems, perhaps causing some of them. True intelligence is a well calibrated “scalar” domain specific problem (uncertainty) reducer. There are few pressing idempotent obstructions in the real world.
Intelligence is the mitigation of uncertainty.
Uncertainty is the domain of negative potential (what,where,why,how?)
Mitigation is the determinant resolve of any constructive or destructive interference affecting (terminal resolve within) the problem domain.
Examples of this may be piled together mountains high, and you may call that functional AGI, though you would be self deceiving. At some point “good enough” may be declared for anything so passing as yourselves.
Basically we need Multimodal LLM's (terrible naming as it's not an LLM then but still).
There's been progress. Look at this 2020 work on neural net controlled drone acrobatics.[1] That's going in the right direction.
[1] https://rpg.ifi.uzh.ch/docs/RSS20_Kaufmann.pdf
I’d be extremely surprised if AI recapitulates the same developmental path as humans did; evolution vs. next-token prediction on an existing corpus are completely different objective functions and loss landscapes.
I then looked it up and they had each copy/pasted the same Stack overflow answer.
Furthermore, the answer was extremely wrong, the language I used was superficially similar to the source material, but the programming concepts were entirely different.
What this tells me is there is clearly no “reasoning” happening whatsoever with either model, despite marketing claiming as such.
Stepping back a level, it may only actually tell us that MRIs measure blood flow.
> Recent work has revealed that the neural activity patterns correlated with sensation, cognition, and action often are not stable and instead undergo large scale changes over days and weeks—a phenomenon called representational drift.
[...]
So, I'm not sure how conclusive this fmri activation study is either.
Though, is there a proto language that's not even necessary for the given measured aspects of condition?
Which artificial network architecture best approximates which functionally specialized biological neutral networks?
OpenCogPrime:KnowledgeRepresentation > Four Types of Knowledge: https://wiki.opencog.org/w/OpenCogPrime:KnowledgeRepresentat... :
> Sensory, Procedural, Episodic, Declarative
From https://news.ycombinator.com/item?id=40105068#40107537 re: cognitive hierarchy and specialization :
> But FWIU none of these models of cognitive hierarchy or instruction are informed by newer developments in topological study of neural connectivity;
ultimately, there's no reason that a general algorithm couldn't do the job of a specific one, just less efficiently.
An easy conclusion to jump to but I believe we need to be more careful. Nothing in these findings proves conclusively that non-verbal reasoning mechanism equivalent to humans couldn't evolve in some part of a sufficiently large ANN trained on text and math. Even though verbal and non-verbal reasoning occurs in two distinct parts of the brain, it doesn't mean they're not related.
You mean besides a few layers of LLMs near input and output that deal with tokens? We have the rest of the layers.
1. Syntax
2. Semantics
3. Pragmatics
4. Semiotics
These are the layers you need to solve.
Saussure already pointed out these issues over a century ago, and Linguists turned ML Researchers like Stuart Russell and Paul Smolensky tried in vain to resolve this.
It basically took 60 years just to crack syntax at scale, and the other layers are still fairly far away.
Furthermore, Syntax is not a solved problem yet in most languages.
Try communicating with GPT-4o in colloquial Bhojpuri, Koshur, or Dogri, let alone much less represented languages and dialects.
Higher order faculties aside, animals seem like us, just simpler.
The higher functioning ones appear to have this missing thing too. We can see it in action. Perhaps all of them do and it is just harder for us when the animal thinks very differently or maybe does not think as much, feeling more, for example.
----
Now, about that thing... and the controversy:
Given an organism, or machine for this discussion, is of sufficiently robust design and complexity that it can precisely differentiate itself from everything else, it is a being.
This thing we are missing is an emergent property, or artifact that can or maybe always does present when a state of being also presents.
We have not created a machine of this degree yet.
Mother nature has.
The reason for emergence is a being can differentiate sensory input as being from within, such as pain, or touch, and from without, such as light or motion.
Another way to express this is closed loop vs open loop.
A being is a closed loop system. It can experience cause and effect. It can be the cause. It can be the effect.
A lot comes from this closed loop.
There can be the concept of the self and it has real meaning due to the being knowing what is of itself or something, everything else.
This may be what forms consciousness. Consciousness may require a closed loop, and organism of sufficient complexity to be able to perceive itself.
That is the gist of it.
These systems we make are fantastic pieces. They can pattern match and identify relationships between the data given in amazing ways.
But they are open loop. They are not beings. They cannot determine what is part of them, what they even are,or anything really.
I am both consistently amazed and dismayed at what we can get LLM systems to do.
They are tantalizingly close!
We found a piece of how all this works and we are exploiting the cral out of it. Ok fine. Humans are really good at that.
But it will all taper off. There are real limits because we will eventually find the end goal will be to map out the whole problem space.
Who has tried computing that? It is basically all possible human thought. Not going to happen.
More is needed.
And that "more" can arrive at thoughts without having first seen a few bazillion to choose from.
An example was the problem of memory shared between systems. ML people started doing LLM’s with RAG. I looked into neuroscience which suggested we need a hippocampus model. I found several papers with hippocampus-like models. Combining LLM’s, vision, etc with hippocampus-like model might get better results. Rinse repeat for these other brain areas wherever we can understand them.
I also agree on testing the architectures with small, animal brains. Many do impressive behaviors that we should be able to recreate in simulators or with robotics. Some are useful, too, like how geese are good at security. Maybe embed a trained, goose brain into a camera system.
I am not convinced it follows. Sure LLMs don’t seem complete however there’s a lot of unspoken inference going on in LLMs that don’t map into a language directly already - the inner layers of the deep neural net that operates on abstract neurons.
Perhaps, but the relative success of trained LLMs acting with apparent generalised understanding may indicate that it is simply the interface that is really an LLM post training;
That the deeper into the network you go (the further from the linguistic context), the less things become about words and linguist structure specifically and the more it becomes about things and relations in general.
(This also means that multiple interfaces can be integrated, sometimes making translation possible, e.g.: image <=> tree<string>)
We have, it's called DreamCoder. There's a paper and everything.
Everything needed for AGI exists today, people simply have (incorrect) legacy beliefs about cognition that are holding them back (e.g. "humans are rational").
https://arxiv.org/abs/2006.08381
Despite being an LLM skeptic of sorts, I don’t think that necessarily follows. The LLM matrix multiplication machinery may well be implementing an equivalent of the human non-language cognitive processing as a side effect of the training. Meaning, what is separated in the human brain may be mixed together in an LLM.
Deleted Comment
You would think the whole "split-brain" thing would have been the first clue; apparently not.
We need to add the 5 senses, of which we have now image, audio and video understanding in LLMs. And for agentic behavior they need environments and social exposure.
Humans not taking this approach doesn’t mean that AI cannot.
And yeah it seems that core primitives of intelligence exist very low in our brains. And with people like Michael Levin, there may even be a root beside nervous systems.
Spoiler alert: brains require a lot of blood, constantly, just to not die. Looking at blood flow on an MRI to determine neural circuitry has to deal with the double whammy of both an extremely crude tool and a correlation/causation fallacy.
This article and the study are arguably useless.
I used to rationalize to myself along similar lines for a long time, then I realized that I'm just not as smart as I thought I was.
I'm brilliant - I've read volumes of encyclopedias, my hobbies include comparative theology, etymology, quantum mechanics and predicting the future with high accuracy (I only mention stuff I'm certain of tho ;) but so much so it disturbs my friends and family.
The highest I scored was in the 160s as a teenager but I truly believe they were over compensating for my age - only as an adult have I learned most children are stupid and they maybe in fact didn't over compensate. I am different than anyone else I've ever personally met - I fundamentally see the world different.
All of that is true but that's a rather flawed way of assessing intelligence - fr. I'm being serious. The things we know can free us as much as they can trap us - knowledge alone doesn't make a man successful, wealthy, happy or even healthy - I'm living evidence of this. That doesn't cut it as a metric for prediction of much. There are other qualities that are far more valuable in the societal sense.
Every Boss I've ever worked for has been dumber than me - each one I've learned invaluable stuff from. I was a boss once - in my day I owned and self taught/created an entire social network much like FB was a few years ago, mine obviously didn't take off and now I'm a very capable bum. Maybe someday something I'm tinkering with will make me millions but prolly not, for many reasons, I could write books if I wanted ;)
At the end of the day, the facts are what they are - there is an optimal level of intelligence that is obviously higher than the bottom but is nowhere near the top tier, very likely near that 100 IQ baseline. What separates us all is our capabilities - mostly stuff we can directly control, like learning a trade.
A Master Plumber is a genius plumber by another name and that can and obviously is most often, learned genius. What you sus about yourself is truth - don't doubt that. No IQ test ever told me I lacked the tenacity of the C average student that would employ me someday - they can't actually measure the extent of our dedicated capacity.
I kno more than most people ever have before or rn presently - I don't know as much about plumbing as an apprentice with 2 years of a trade school dedicated to plumbing and a year or two of experience in the field, that's the reality of it. I could learn the trade - I could learn most every trade, but I won't. That's life. I can tell you how you the ancients plumbed bc that piqued my curiosity and I kno far more about Roman plumbing than I do how a modern city sewer system works. That's also life.
It isn't what we kno or how fast we can learn it - it's what we do that defines us.
Become more capable if you feel looked down on - this is the way bc even if what you hone your capabilities of can be replicated by others most won't even try.
That's my rant about this whole intelligence perception we currently have as a society. Having 100 IQ is nowhere near the barrier that having 150 IQ is.
Rant aside, to the article - how isn't this obvious? I mean feelings literally exist - not just the warm fuzzy ones, like the literal feeling of existence. Does a monkey's mind require words to interpret pain or pleasure for example. Do I need to know what "fire" or "hot" is in a verbal context to sufficiently understand "burn" - words exists to convey to to others what doesn't need to be conveyed to us. That's their function. Communication. To facilitate communication with our social brethren we adopt them fundamentally as our Lego blocks for understanding the world - we pretend that words comprising language are the ideas themselves. A banana is a - the word is the fruit, they are the same in our minds but if I erase the word banana and all it's meaning of the fruit and I randomly encounter a banana - I still can taste it. No words necessary.
Also, you can think without words, deliberately and consciously - even absentmindedly.
And LLMs can't reason ;)
Truthfully, the reality is that a 100 IQ normal human is far more capable than any AI I've been given access to - in almost every metric I attempted to asses I ultimately didn't even bother as it was so obvious that humans are functionally superior.
When AI can reason - you, and everyone else, will kno it. It will be self evident.
Anyways, tldr: ppl are smarter than given credit for, smarter and much more capable - IQ is real and matters but far less than we are led to believe. People are awesome - the epitome of biological life on Earth and we do a lot of amazing things and anyone can be amazing.
I hate it when the Hacker News collective belittles itself - don't do that. I rant here bc it's one of the most interesting places I've found and I care about what all of you think far more than I care about your IQ scores.
The abstract visualizations I could build in my mind where comparable to semi-transparent buildings that I could freely spin, navigate and bend to connect relations.
In my mid-twenties, someone introduced me to the concept of people using words for mental processes, which was completely foreign to me up to this point.
For some reason, this made my brain move more and more towards this language-based model and at the same time, I felt like I was losing the capacity for complex abstract thoughts.
Still to this day I (unsuccessfully) try to revive this and unlearn the language in my head, which feels like it imposes a huge barrier and limits my mental capacity to the capabilities of what the language my brain uses at the given time (mostly EN, partially DE) allows to express.
I think that I ultimately developed an obsessive need to cite all my ideas against the literature and formulate natural language arguments for my claims to avoid being bludgeoned over the head with wordcelry and being seen as inferior for my lesser verbal fluency despite having written software for years at that point, since early childhood, and even studied computer science.
Basically what to most people is so obvious that it becomes transparent ("air") isn't to us, which apparently is an incredible gift for becoming a language researcher. Or a programmer.
It seems more like a complement to it: the idea arises, and then I have this compulsion to verbalise it, which gets quite frustrating as it takes several iterations. Clearly words do matter to me as a way to structure and record my ideas but there is something that pre-empts verbalisation and to some extent resists it.
I cannot provide insight on how I arrive at ideas. Even when I did literary criticism, the best I can say is that I absorbed lots of text and then suddenly a pattern would spring out. But the same things would happen for me studying maths or the hard sciences.
Software engineering is actually a bit different for me because I am not naturally a good algorithmic problem solver. Really I am somebody very passionate about computing who has a near-compulsion to see and collect more and more technology. So for me it is as simple as saying "this resembles a reader monad" or "this puns on the active record pattern". Less impressive than my humanities intelligence but worth maybe 10x the amount in the labour market :-)
This begs a question though: Since programming is mostly done with language - admittedly primitive/pidgin ones - why isn't that a struggle? Not sure if you're a programmer yourself, but if so do you prefer certain programming languages for some sense of "less-verbalness" or does it even matter?
Just wondering, not attacking your claim per se.
Parent isn't saying they can't handle language (and we wouldn't have this discussion in the first place), just that they better handle complexity and structure in non verbal ways.
To get back to programming, I think this do apply to most of us. Most of us probably don't think in ruby or JS, we have a higher vision of what we want to build and "flatten" it into words that can be parsed and executed. It's of course more obvious for people writing in say basic or assembly, some conversion has to happen at some point.
> The dog's owner's house's roof's angle's similarity to an equilateral triangle is remarkable.
I very strongly suspect that you're overestimating yourself.
I think this is completely wrong-headed. It's like saying that until cars came about we just didn't have anything other than animals that could move around under its own power, therefore in order to understand how animals move around we should go and study cars. There is a great gulf of unsubstantiated assumptions between observing the behaviour of a technological artifact, like a car or a statistical language model, and thinking we can learn something useful from it about human or more generally animal faculties.
I am really taken aback that this is a serious suggestion: study large language models as in-silico models of human linguistic ability. Just putting it down in writing like that rings alarm bells all over the place.
It's hard for me to understand where my peers are coming from on the other side of this argument and respond without being dismissive, so I'll do my best to steelman the argument later.
Machine learning models are function approximators and by definition do not have an internal experience distinct from the training data any more than the plus operator does. I agree with the sentiment that even putting it in writing gives more weight to the position than it should, bordering on absurdity.
I suppose this is like the ELIZA phenomena on steroids, is the only thing I can think of for why such notions are being entertained.
However, to be generous, lets do some vigorous hand waving and say we could find a way to have an embodied learning agent gather sublinguistic perceptual data in an online reinforcement learning process, and furthermore that the (by definition) non-quantifiable subjective experience data could somehow be extracted, made into a training set, and fit to a nicely parametric loss function.
The idea then is that could find some architecture that would allow you to fit a model to the data.
And voila, machine consciousness, right? A perfect model for sentience.
Except for the fact that you would need to ignore that in the RL model gathering the data and the NN distilled from it, even with all of our vigorous hand waving, you are once again developing function approximators that have no subjective internal experience distinct from the training data.
Let's take it one step further. The absolute simplest form of learning comes in the form of habituation and sensitization to stimuli. Even microbes have the ability to do this.
LLMs and other static networks do not. You can attempt to attack this point by fiatting online reinforcement learning or dismissing it as unnecessary, but I should again point out that you would be attacking or dismissing the bare minimum requirement for learning, let alone a higher order subjective internal experience.
So then the argument, proceeding from false premises, would claim that the compressed experience in the NN could contain mechanical equivalents of higher order internal subjective experiences.
So even with all the might vigorous hand waving we have allowed, you have at best found a way to convert internal subjective processes to external mechanical processes fit to a dataset.
The argument would then follow, well, what's the difference? And I could point back to the microbe, but if the argument hasn't connected by this point, we will be chasing our tails forever.
A good book on the topic that examines this in much greater depth is "The Self Assembling Brain".
https://a.co/d/1FwYxaJ
That being said, I am hella jealous of the VC money that the grifters will get for advancing the other side of this argument.
For enough money I'd probably change my tune too. I can't by a loaf of bread with a good argument lol
I could enter what we all here call the "Zone" quite often when i was young (once while doing math :D). I still can, but rarely on purpose, and rarely while coding. I have a lot of experience in this state, and i can clearly say that a marker of entering the zone is that your thoughts are not "limited" by language anymore and the impression of clarity and really fast thinking. This is why i never thought that language was required for thinking.
Now the question: would it be possible to scan the brain of people while they enter the zone? I know it isn't a state you can reach on command, but isn't it worth to try? understand the mechanism of this state? And maybe understand where our thought start?
https://en.wikipedia.org/wiki/Flow_(psychology)
That is, until the code refuses to work. Then the code is a bitch and I need a break.
> Language serves not only to express thoughts, but to make possible thoughts which could not exist without it. It is sometimes maintained that there can be no thought without language, but to this view I cannot assent: I hold that there can be thought, and even true and false belief, without language. But however that may be, it cannot be denied that all fairly elaborate thoughts require words.
> Human Knowledge: Its Scope and Limits by Bertrand Russell, Section: Part II: Language, Chapter I: The Uses of Language Quote Page 60, Simon and Schuster, New York.
Of course, that would contravene the popular narrative that philosophers are pompous idiots incapable of subtlety.
Practically, I think the origins of fire-making abilities in humans tend to undermine that viewpoint. No other species is capable of reliably starting a fire with a few simple tools, yet the earliest archaeological evidence for fire (1 mya) could mean the ability predated complex linguistic capabilities. Observation and imitation could be enough for transmitting the skill from the first proto-human who successfully accomplished the task to others.
P.S. This is also why Homo sapiens should be renamed Homo ignis IMO.
It’s doubtless to me that thinking happens without intermediary symbols; but it’s also obvious that I can’t think deeply without the waypoints and context symbols provide. I think it is a common sense opinion.
Just a few days ago was "What do you visualize while programming?", and there's a few of us in the comments that, when programming, think symbolically without language: https://news.ycombinator.com/item?id=41869237
> You can ask whether people who have these severe language impairments can perform tasks that require thinking. You can ask them to solve some math problems or to perform a social reasoning test, and all of the instructions, of course, have to be nonverbal because they can’t understand linguistic information anymore.
Surely these "non-verbal instructions" are some kind of language. Maybe all human action can be considered language.
A contrarian example to this research might be feral children, i.e people who have been raised away from humans.[0] In most cases they are mentally impaired; as in not having human-like intelligence. I don't think there is a good explanation why this happens to humans. And why it doesn't happen to other animals, which develop normally in species-typical way whether they are in the wild or in human captivity. It seems that most human behavior (even high-level intelligence) is learned / copied from other humans, and maybe this copied behavior can be considered language.
If humans are "copy machines", there's also a risk of completely losing the "what's it like to be a human" behavior if children of the future are raised by AI and algorithmic feeds.
[0] https://en.wikipedia.org/wiki/Feral_child
> DA was impaired in solving simple addition, subtraction, division or multiplication problems, but could correctly simplify abstract expressions such as (b×a)÷(a×b) or (a+b)+(b+a) and make correct judgements whether abstract algebraic equations like b − a = a − b or (d÷c)+a=(d+a)÷(c+a) were true or false.
> Sensitivity to the structural properties of numerical expressions was also evaluated with bracket problems, some requiring the computation of a set of expressions with embedded brackets: for example, 90 [(3 17) 3].
Discussions of whether or not these sorts of algebraic or numerical expressions constitute a "language of mathematics" aside (despite them not engaging the same brain regions and structures associated with the word "language"); it may be the case that these sorts of word sequences and symbols processed by structures in the brain's left hemisphere are not essential for thought, but can still serve as a useful psychotechnology or "bicycle of the mind" to accelerate and leverage its innate capabilities. In a similar fashion to how this sort of mathematical notation allows for more concise and precise expression of mathematical objects (contrast "the number that is thrice of three and seventeen less of ninety") and serves to amplify our mathematical capacities, language can perhaps be seen as a force multiplier; I have doubts whether those suffering from aphasia or an agrammatic condition would be able to rise to the heights of cognitive performance.
[0] https://pubmed.ncbi.nlm.nih.gov/17306848/
[1] https://pubmed.ncbi.nlm.nih.gov/15713804/