The confabulation to justify picking out related images that the left brain never observed (chicken and snow shovel in the article) reminds me profoundly of the confident slop produced by LLMs. Make you wonder if llms might be one half of the "brain" of a true AGI
More then half! Human experience is more confabulation than not. The optic nerve's digital-equivalent bandwidth is an estimated 10Mbps, predominantly dedicated to a narrow radius around the center of vision. Everything around the outside of your vision is a few fuzzy pixels that are in-filled with plausible data. Same goes for the blind spot created by the optic disc, which is actually fully fabricated as it has no cones or rods at all!
Key point here is "looks like" I suggest if you want to argue this further to invest the time asking Brain Scientists what they think. Not AI scientists but people who actually work in cognition.
The confidence seems to be an artifact of fine tuning. The first instruction trained models were given data sets with answers to questions but generally omitted non answers to things the model didn't know.
Later research showed that models know that they don't know certain pieces of information, but the fine tuning constraint of providing answers did not give them the ability to express that they didn't know.
Asking the model questions against known information can produce a correct/incorrect map detailing a sample of facts that the model knows and does not know. Fine tuning a model to say "I don't know" in response to the those questions where it was incorrect can allow it to generalise the concept to its internal concept of unknown.
It is good to keep in mind that the models we have been playing with are just the first ones to appear. GPT 3.5 is like the Atari 2600. You can get it provide a limited experience for what you want and its cool that you can do it at all, but it is fundamentally limited and far from an ideal solution. I see the current proliferation of models to be like the Cambrian explosion of early 8 bit home computers. Exciting and interesting technology which can be used for real world purposes, but you still have to operate with the knowledge of the limitations forefront in your mind and tailor tasks to allow them to perform the bits they are good at. I have no-idea of the timeframe, but there is plenty more to come. There have been a lot of advances revealed in papers. A huge number of those advances have not yet coalesced into shipping models. When models cost millions to train you want to be using a set of enhancements that play nicely together. Some features will be mutually exclusive. By the time you have analysed the options to find an optimal combination, a whole lot of new papers will be suggesting more options.
We have not yet got the thing for AI that Unix was for computers. We are just now exposing people to the problems that drives the need to create such a thing.
I believe most confident statements people make, are established the same way. There are some anchor points (inputs and vivid memories) and some part of the brain in some stochastic way dreams up connections. Then we convince ourselves that the connections are correct, just because they match some earlier seen pattern or way of reasoning.
The fact that the explaining part of the brain fills in any blanks in a creative manner (you need the shovel to clean the chicken shed), reminds me to some replies of LLMs.
I once provided an LLM the riddle of the goat, cabbage and wolf, and changed the rules a bit. I prompted that the wolf was allergic to goats (and hence would not eat them). Still the llm insisted on not leaving them together on the same river bank, because the wolf would otherwise sneeze and scare the goat away.
My conclusion was that the llm solved the riddle using prior knowledge plus creativity, instead of clever reasoning.
I believe LLMs are entirely analogous to the speech areas of the brain. They have a certain capacity of speaking automatically, reflexively, without involving (other) memory for example. That is how you are able to deliver quippy answers, that is where idioms "live". You can see this in people with certain kinds of brain damage, if they are unable to recall certain memories (or sometimes if you press somebody to recall memories that they don't have) they will construct elaborate stories on the spot. They won't even notice that they are making it up. This is called confabulation, and I think it is a much better term than hallucination for what LLMs do when they make up facts.
I feel this analogy confirmed by the fact that chain of thought works so well. That is what (most?) people do when they actively "think" about a problem. They have a kind of inner monologue.
Now, we have already reached the point that LLMs are much smarter than the language areas of humans - but not always smarter than the whole human. I think the next step towards AGI would be to add other "brain areas". A limbic system that remembers the current emotion and feeds it as an input into the other parts. We already have dedicated vision and audio AIs. Maybe we also need a system for logical reasoning.
I always thought it was interesting that the human brain grew relatively quickly in evolutionary history. 3 million years ago, our ancestors had a 400 cc brain. 2.5 million years later, it was 1,400 ccs--more than 3 times larger.
That implies to me that a larger brain immediately benefited our ancestors. That is, going from 400 to 410 ccs had evolutionary advantage and so did 410 to 420, etc.
That implies that once the brain architecture was set, you could increase intelligence through scale.
I bet there are some parallels to current AI there.
This comment reminded me of "A Thousand Brains: A New Theory of Intelligence" by Jeff Hawkins, which explores this. To Hawkins, the brain's relatively fast evolution implies there's a general-purpose "compute" unit that, once it evolved once, could proliferate without novel evolutionary design. He claims this unit is the brain's cortical column, and provides a lot of interesting evidence and claims that I no longer remember :)
>Miller’s study uses a test called the “trait-judgment task”: A trait like happy or sad flashes on a screen, and research subjects indicate whether the trait describes them. Miller has slightly modified this task for his split-brain patients—in his experiments, he flashes the trait on a screen straight in front of the subject’s gaze, so that both the left and right hemispheres process the information. Then, he quickly flashes the words “me” and “not me” to one side of the subject’s gaze—so that they’re processed only by one hemisphere—and the subject is instructed to point at the trait on the screen when Miller flashes the appropriate descriptor.
Seems to me (not a neuroscientist) like there's a flaw in that experiment: how would the right hemisphere understand the meaning of the words, if language is only processed by the left? I also recall reading that the more "primitive" parts of our brains don't have a concept of negation.
But maybe they have been considering this and it's no issue?
Readit News
https://archive.ph/gJ32A
(Not a brain scientist btw)
Later research showed that models know that they don't know certain pieces of information, but the fine tuning constraint of providing answers did not give them the ability to express that they didn't know.
Asking the model questions against known information can produce a correct/incorrect map detailing a sample of facts that the model knows and does not know. Fine tuning a model to say "I don't know" in response to the those questions where it was incorrect can allow it to generalise the concept to its internal concept of unknown.
It is good to keep in mind that the models we have been playing with are just the first ones to appear. GPT 3.5 is like the Atari 2600. You can get it provide a limited experience for what you want and its cool that you can do it at all, but it is fundamentally limited and far from an ideal solution. I see the current proliferation of models to be like the Cambrian explosion of early 8 bit home computers. Exciting and interesting technology which can be used for real world purposes, but you still have to operate with the knowledge of the limitations forefront in your mind and tailor tasks to allow them to perform the bits they are good at. I have no-idea of the timeframe, but there is plenty more to come. There have been a lot of advances revealed in papers. A huge number of those advances have not yet coalesced into shipping models. When models cost millions to train you want to be using a set of enhancements that play nicely together. Some features will be mutually exclusive. By the time you have analysed the options to find an optimal combination, a whole lot of new papers will be suggesting more options.
We have not yet got the thing for AI that Unix was for computers. We are just now exposing people to the problems that drives the need to create such a thing.
Seems pertinent, and now I will try to read it again. Perhaps it will be useful for reference by others.
I once provided an LLM the riddle of the goat, cabbage and wolf, and changed the rules a bit. I prompted that the wolf was allergic to goats (and hence would not eat them). Still the llm insisted on not leaving them together on the same river bank, because the wolf would otherwise sneeze and scare the goat away.
My conclusion was that the llm solved the riddle using prior knowledge plus creativity, instead of clever reasoning.
I feel this analogy confirmed by the fact that chain of thought works so well. That is what (most?) people do when they actively "think" about a problem. They have a kind of inner monologue.
Now, we have already reached the point that LLMs are much smarter than the language areas of humans - but not always smarter than the whole human. I think the next step towards AGI would be to add other "brain areas". A limbic system that remembers the current emotion and feeds it as an input into the other parts. We already have dedicated vision and audio AIs. Maybe we also need a system for logical reasoning.
That implies to me that a larger brain immediately benefited our ancestors. That is, going from 400 to 410 ccs had evolutionary advantage and so did 410 to 420, etc.
That implies that once the brain architecture was set, you could increase intelligence through scale.
I bet there are some parallels to current AI there.
Seems to me (not a neuroscientist) like there's a flaw in that experiment: how would the right hemisphere understand the meaning of the words, if language is only processed by the left? I also recall reading that the more "primitive" parts of our brains don't have a concept of negation.
But maybe they have been considering this and it's no issue?
https://www.youtube.com/watch?v=3V3_Y_FuMYk